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EXECUTIVE  SUMMARY 


PROBLEM 

Each  year,  the  Government  spends  billions  of  dollars 
on  engineering  changes,  oontnct  daims,  and  modifications 
to  recently  purchased  items.  In  many  cases,  these  costs 
could  have  been  avoided  if  weaknesses  in  the  specifications 
were  found  and  fixed  before  entering  into  contracts. 

OBJECTIVE 

The  techniques  developed  under  this  Independent 
Exploratoiy  Development  are  intended  to  help  writers 
and  reviewers  of  specifications  identify  such  weaknesses 
quickly,  thoroughly,  and  accurately. 

APPROACH 

There  were  two  parts  to  the  effort  on  this  project.  The 
first  involved  a  ccmtinued  search  for  literature  that  rdates 
to  specification  writing.  It  also  involved  gafiiering  examples 
of  qjecification  errors  by  examining  drafts  of  qxdfications. 
This  "knowledge-engineering"  was  needed  in  order  to 
learn  more  about  what  makes  specifications  good  or 
bad,  so  that  the  computer  might  be  programmed  to  flag 
significant  sources  of  potential  error. 

The  other  domain  was  experimentation  with  software 
techniques  for  automating  the  examination  process. 
It  has  been  centered  around  the  development  of  a  small- 
scale  natural  language  parser  intended  ftrr  parsing  sentences 
that  are  typical  in  training  systems  specifications  and 
statements  of  work. 

HNDINGS 

Surprisingly,  for  such  an  important  subject  as  it  is,  diere 
is  relatively  little  organized  knowledge  about  how  to 
write  and  edit  qxdfications.  Much  material  was  borrowed 
from  other  fidds,  and  some  of  it  is  more  useful  to  human 
reviewers  than  to  electronic  ones.  A  great  deal  of  the 


knowledge  gathered  consists  of  the  awareness  of  many 
linguistic  minutiae,  any  one  of  which  could  lead  to  a 
jerious  contract  dispute. 

We  have  found  that  a  relatively  sinq>le  parser  yields 
information  on  sentence  structure  sufficimt  to  generate 
reasonably  accurate  warning  messages  about  certain 
error-prooe  fypes  of  syntax.  In  additioo,  we  have  fcund 
that  data  extracted  by  the  parser  can  be  used  to  facilitate 
checking  the  oitire  document  for  explicitly  conflicting 
requirements. 

CONCLUSIONS 

Ai^xndix  C  of  this  report  sets  a  standard  for  eliminarioo 
of  Ifae  fypes  of  eiiots  oomnonly  encountered  in  specificalions. 
In  doing  so,  it  provides  information  not  furnished  by 
the  existing  iqrplicable  standards,  MIL-STD-490Aand 
MILrSn>961C.  Considering  the  stringency  of  Appendix 
C,  one  may  readily  conclude  that  writing  high-quality 
specifications  is  a  task  that  requires  an  uncommonly 
high  levd  of  literacy,  a  great  deal  of  specialized  training, 
and  the  help  of  cognitive  tools  like  those  developed  on 
this  project. 

RECOMMENDATIONS 

In  light  of  what  has  been  learned  on  this  project,  it  is 
recommended  that  the  knowledge-oigineering  effort 
r^arding  specification  writmg  be  conducted  on  a  continuitig 
basis.  It  is  furdier  recommended  that  additional  research 
be  conducted  to  appfy  emeigii^  todmclcgy  to  foe  specificifion 
quality  problem.  Finally,  it  is  recommended  that  the 
parser  software  developed  under  this  effort  be  ported 
to  the  Windows  enviromiMnt  so  that  ^tactic  ambiguify 
warnings  can  be  made  available  to  users  of  the  Joint 
Acquisition  Managemoit  System  (JAMS). 
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Engineering  Specification 
Editing  Tools 


I .  IMTRODnCTIOH 


Problem 

The  process  of  purchasing  goods  and  services  under  contract  is 
lengthy  and  complicated  for  many  reasons.  In  the  course  of  going 
from  the  need  to  the  delivered  goods  and  services,  a  great 
quantity  of  written  matter  is  produced.  All  this  paperwork 
serves  the  purpose  of  communicating  information  to  a  large  number 
of  people  with  a  wide  variety  of  responsibilities.  Among  the 
most  important  procurement  documents  are  those  that  describe  in 
detail  the  goods  and  services  to  be  delivered  (Richardson,  1990) . 
These  are  engineering  drawings,  engineering  specifications,  and 
statements -of -work.  Specifications  and  statements -of -work  nearly 
always  consist  of  sparsely  illustrated  text.  They  may  or  may  not 
be  accompanied  by  a  set  of  drawings.  For  the  purposes  of  this 
discussion,  the  text  portion  of  these  docviments  will  be  referred 
to  collectively  as  specifications.  The  specifications  discussed 
here  are  informal  specifications.  There  is  no  intention  for  them 
to  be  as  complete  and  rigorous  as  the  formal  specifications 
discussed  by  computer  scientists.  They  are  intended  to  describe 
only  those  features  of  the  end  item  that  bear  upon  fitness  for 
its  intended  purpose . 

Specifications  are  like  any  other  work  product.  They  are  the 
result  of  applying  lcd>or  to  raw  materials,  producing  a  new  item 
with  added  value.  They  are  subject  to  defects  just  like  other 
products.  Where  they  differ  from  most  other  products  is  that,  in 
addition  to  being  products  themselves,  they  describe  products  as 
well.  In  the  language  of  the  computer  scientist,  they  are  "meta¬ 
products"  . 

Each  defect  in  specification  will  result  in  one  of  five  possible 
events .  These  are ; 

a.  The  error  will  go  unnoticed  or  will  be  gratuitously 
corrected  by  the  reader,  and  have  no  effect  on  the  end  item. 

b.  The  error  will  be  embodied  in  the  end  item,  causing  it  to  be 
unfit  for  its  intended  purpose,  though  not  defective  under 
the  terms  of  the  contract. 

c.  The  error  will  conflict  with  another  portion  of  the 
specification,  creating  a  situation  known  as  "impossibility 
of  performance." 
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d.  The  error  will  cause  the  end  item  to  have  an  unneeded,  but 
otherwise  harmless  feature,  and  be  lacking  a  needed  one. 

e.  The  error  will  be  found  and  formally  corrected  after 
contract  award. 

The  final  result  of  all  outcomes  except  the  first  is  cost  growth. 
In  practice,  the  majority  of  defects  lead  to  the  first  outcome. 
This  is  fortunate,  since  such  defects  are  far  more  common  than 
one  might  expect.  In  reviewing  numerous  specifications,  it  is 
not  unusual  to  find  at  least  three  or  four  defects  per  typed 
page.  While  this  number  seems  appalling,  the  author  finds  it 
encouraging  that  so  many  contractors  exercise  goodwill  in  dealing 
with  specification  errors  rather  than  using  them  to  financial 
advantage  whenever  possible.  Usually  such  errors  become  problems 
only  when  the  project  is  beset  by  financial  or  technical 
difficulties.  At  such  time,  their  effect  is  to  make  the  contract 
virtually  unenforceable. 


Purpose 

The  information  gathered  and  techniques  developed  under  this 
Independent  Exploratory  Development  (lED)  project  are  intended  to 
help  writers  of  engineering  specifications  identify  defects  in 
their  work  quickly,  thoroughly,  and  accurately,  so  that  they  may 
be  corrected  as  early  as  possible. 


Background 

Literature  covered  on  this  topic  comes  under  six  subject 
headings : 

a.  Rhetoric, 

b.  Technical  writing, 

c.  Legal  writing, 

d.  Engineering  specifications, 

e.  Natural  language  processing,  and 

f.  Philosophy  of  science. 

There  is  a  large  body  of  widely  read  literature  dealing  with  the 
skill  of  expressing  oneself  well,  both  on  general  rhetoric  and  on 
technical  writing.  Many  of  the  types  of  errors  found  in 
specifications  are  addressed  in  it  (Bly  and  Blake,  1982,  Strunk 
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and  White,  1979) .  But  the  task  of  drafting  specifications 
requires  knowledge  beyond  that  level.  Books  on  legal  writing 
discuss  some  topics  that  are  very  important  to  specification 
writers,  like  the  correct  use  of  commas  (Block,  1986) ,  but  they 
aim  to  cover  a  broader  subject  than  specification  writers  need, 
and  pay  little  attention  to  the  many  sources  of  ambiguity  in  the 
English  language  that  are  of  important  concern  in  specifications. 


Literature  on  specifications  per  se  comes  from  two  discourse 
communities;  the  construction  industry  and  the  systems  management 
community.  Despite  the  fact  that  a  difference  between  the  two  is 
sometimes  indicated  by  authors  from  both  camps,  there  is  no 
genuine  need  to  make  such  a  distinction.  Contracts  for  civil 
engineering  works  are  governed  by  the  same  principles  as 
contracts  for  any  other  kind  of  engineering.  Likewise,  civil 
engineers  deal  with  the  Scime  laws  of  nature  and  the  same  economic 
realities  as  all  other  engineers.  Consequently,  literature  from 
both  sources  should  be  regarded  equally. 

On  the  whole,  they  both  tend  to  lean  towards  discussion  of  what 
might  be  called  macrostructure  -  format,  organization,  sources  of 
information,  types  of  requirements,  mandatory  requirements,  and 
other  topics  not  related  directly  to  the  words  and  sentences  from 
which  actual  documents  are  built.  They  usually  talk  only  briefly 
of  microstructure,  covering  particularly  the  distinction  between 
the  meanings  of  the  words  "will"  and  "shall".  A  notable 
exception  to  this  brevity  is  found  in  Henry  Henkin's  book 
Drafting  Engineering  Contracts  (1988) ,  which  devotes  about  forty 
pages  to  microstructure.  Some  of  Henkin's  examples  illustrate 
the  very  same  phenomena  of  the  English  language  that  are  well- 
covered  in  literature  on  automatic  processing  of  natural 
language,  which  was  examined  initially  for  information  on  how  to 
program  the  parser.  Hence  the  review  of  literature  from  that 
source  (e.g.  Winograd,  1983)  was  expanded  in  search  of  more 
examples  and  better  explanations  of  the  sources  of  ambiguity  in 
the  English  language. 

The  pursuit  of  absolute  rigor  in  the  definition  of  terminology, 
because  of  its  importance  in  the  natural  and  social  sciences,  has 
been  investigated  at  great  length  by  some  of  the  leading 
scientists  of  our  times  (Hempel,  1966) ,  and  published  under  the 
subject  heading  "Philosophy  of  Science."  Certain  topics  from 
that  work  are  highly  applicable  to  the  specification  problem, 
notably  Bridgman's  concept  of  operational  definitions. 
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Organization  of  this  Report 

Section  II  is  based  on  an  earlier  document  that  has  been  updated 
to  reflect  the  information  collected  in  the  course  of  the 
project.  It  provides  a  description  of  the  types  of  errors  that 
may  be  found  in  specifications,  and  theorizes  about  conditions 
that  may  contribute  to  their  likelihood.  Appendix  A  contains  a 
more  detailed  hierarchy  of  the  error  types  described.  Section 
III  is  a  description  of  the  exploratory  software  development 
performed  as  part  of  the  project,  and  Section  IV  discusses  some 
theoretical  aspects  of  specifying  that  were  encountered. 

Appendix  B  is  a  detailed  list  of  words,  phrases  and  syntax  that 
are  likely  to  be  associated  with  specification  errors.  Appendix 
C  is  a  guide  for  specification  writers  based  on  the  material 
presented  in  Section  I. 
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II.  TYPES  OF  ERRORS  AMD  THEIR  CAUSES 


Developing  a  nomenclature  for  errors 

Our  research  on  specification  defects  started  out  as  an  effort  to 
develop  computer  software  that  would  provide  some  assistance  to 
writers  of  engineering  specifications.  The  aim  was  to  shorten 
the  time  needed  to  develop  a  set  of  specifications  while 
enhancing  its  quality  as  well.  While  much  has  been  accomplished 
along  those  lines,  it  has  become  evident  that  there  are  many 
factors  affecting  specification  quality  that  require  remedies 
other  than  those  that  can  be  provided  by  computer  software.  The 
purpose  of  this  report,  therefore,  is  to  document  what  has  been 
learned  about  specification  defects,  as  well  as  to  discuss 
computer  software  that  may  help  editors  to  recognize  and  correct 
some  of  them. 

To  design  software  that  would  help  locate  errors  in  draft 
specifications,  it  is  first  necessary  to  know  what  types  of 
errors  occur  in  such  text.  In  an  attempt  to  make  such  a 
determination,  drafts  of  engineering  specifications  from  a 
variety  of  sources  have  been  examined.  From  the  data  gathered, 
the  types  of  errors  likely  to  be  found  in  specifications  seem 
more  or  less  consistent,  regardless  of  the  document's  place  of 
origin.  Furthermore,  the  types  of  errors  described  below  have 
been  found  to  occur  almost  as  frequently  in  documents  that  are 
already  released  and  in  use  as  they  do  in  drafts.  In  an  effort 
to  document  the  knowledge  gained,  a  list  was  made  of  26  types  of 
defects  commonly  found  in  specifications.  These  types  were 
grouped  into  seven  major  categories.  Each  of  the  categories  is 
described  briefly  in  the  following  paragraphs.  A  complete  list 
of  the  26  types  of  defects  is  given  in  outline  form.  For  each 
type  of  defect,  a  list  was  made  of  likely  reasons  why  the 
development  process  may  produce  such  a  defective  result.  Many  of 
the  reasons  apply  to  more  than  one  type  of  error.  When 
summarized,  the  list  consists  of  22  factors  affecting  the  quality 
of  engineering  specifications.  Both  complete  lists  are 
represented  jointly  in  Table  I.  The  information  presented  in  the 
lists,  though  more  detailed  and  organized  differently,  closely 
parallels  the  experience  documented  by  McRobb  (1989) . 


Categories  of  Errors 

The  sections  that  follow  are  brief  descriptions  of  seven  general 
classes  into  which  the  26  commonly  found  types  of  defects  may  be 
grouped.  When  considering  a  specific  error  observed  in  practice, 
it  was  sometimes  difficult  to  decide  how  the  error  should  be 
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categorized,  since  many  observed  errors  could  fall  into  more 
than  one  category.  To  that  extent  the  grouping  is  svibjective. 


a.  Organization  Errors 

Organization  errors  are  defects  that  would  prevent  an 
experienced  reader  from  locating  a  specific  piece  of  sought- 
after  information  in  the  document.  Since  nearly  all 
specifications  are  organized  according  to  a  standardized 
outline  like  those  given  in  MIL-STD-490A  (AFSC,  1985) ,  there 
are  only  two  ways  to  make  an  organization  error.  They  are  by 
stating  a  given  requirement  in  the  wrong  place  and  by  numbering 
the  paragraphs  incorrectly.  Organization  errors  may  not  be 
serious  in  themselves,  but  they  contribute  to  the  likelihood  of 
inconsistency  errors  and  injure  the  specifier's  credibility. 
They  also  create  confusion  when  each  person  working  on  the 
project  has  access  only  to  the  portion  of  the  specifications 
that  are  deemed  by  "higher-ups"  to  be  relevant  to  his  or  her 
part  of  the  job.  Such  practice  is  common  in  industry,  and  is 
mandated  by  security  rules  on  sensitive  work. 


b.  Reference  Errors 

Reference  errors  are  cases  where  the  text  refers  to  another 
paragraph,  an  illustration,  table,  another  docinnent,  or  an 
acronym  that  cannot  be  readily  found  by  the  reader.  The 
referenced  item  may  be  inaccessible  for  a  number  of  reasons. 

The  most  common  type  of  reference  error  occurs  when  a  document 
cites  one  of  its  own  paragraphs.  The  reference  starts  out 
being  correct,  but  goes  awry  when  part  of  the  document  is  moved 
and  renumbered,  as  so  often  happens  during  development.  Also 
in  this  category  are  cases  where  the  referenced  document  does 
not  contain  the  information  cited.  A  computer  program  called 
PARANA  (Oriel,  1989)  is  available  to  facilitate  the  location 
and  correction  of  reference  errors. 

c.  Incorrect  Requirements 

Several  times,  requirements  have  been  found  that,  upon 
investigation,  turned  out  to  be  incorrect  in  substance.  They 
were  identified  as  belonging  to  four  categories; 

(1)  Text  wrong  for  logical  reasons, 

(2)  Text  wrong  for  practical  reasons. 
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(3)  Violations  of  law  or  regulations,  and 

(4)  Numbers  or  units -of -measure  wrong. 

Incorrect  requirements  of  the  first  three  types  may  be  extreme 
cases  of  incorrectly  expressed  requirements.  In  these  cases, 
what  was  inadvertently  expressed  seems  plausible  as  something 
the  writer  might  have  actually  wanted  to  specify. 


d.  Incorrectly  E3q>res8ed  Requirements 

One  of  the  most  common  kinds  of  errors  found  in  the  specifica¬ 
tions  studied  was  failure  to  state  the  requirements  clearly, 
correctly,  and  concisely.  While  the  project  engineers 
responsible  for  specifications  are  usually  much  better  writers 
than  many  other  types  of  professionals,  skill  at  clearly 
describing  complex  equipment  seems  to  be  rare.  Furthermore, 
since  most  of  the  text  had  ostensibly  been  edited,  one  may  also 
conclude  that  the  errors  were  missed  by  more  than  one  reviewer. 
Hence  it  is  possible  either  that  the  reviewers  are  not 
recognizing  the  defects,  or  that  the  text  had  not  been  edited 
as  claimed.  Most  likely,  what  happened  was  a  combination  of 
both.  Editing  docximents  of  such  a  nature  as  engineering 
specifications  is  tedious  work,  even  with  the  help  of  state-of- 
the-art  computerized  assistance.  It  requires  specialized 
skills  possessed  by  relatively  few  engineers,  as  well  as  the 
time  and  patience  to  do  the  job  well.  Editing  specifications 
may  also  require  the  objectivity  of  a  person  not  encumbered  by 
preconceived  ideas  about  the  specified  items.  One  difficulty 
in  employing  such  persons  as  editors,  however,  lies  in  the  fact 
that  such  a  person  is  usually  intimidated  by  the  technical 
nature  of  the  text  and  is  reluctant  to  criticize  it. 

A  major  cause  of  incorrectly  expressed  requirements  is  the 
misuse  of  certain  words  and  phrases.  Considerable  progress  has 
been  made  towards  reducing  errors  in  word  usage  by  the  use  of  a 
specialized  personal  computer  program  called  SpecTrE.  SpecTrE 
checks  the  text  for  strings  belonging  to  a  prescribed  set  of 
commonly  misused  words  and  phrases.  Appendix  B  contains  a  list 
of  such  words  and  phrases.  Enhancing  the  performance  of 
programs  like  SpecTrE  and  finding  new  methods  that  they  might 
use  was  the  main  aim  of  this  exploratory  development  project. 
Figure  1.  illustrates  the  relative  frequencies  of  misuse  of 
some  words  that  are  often  found  in  specifications.  The 
frequencies  vary  considerably  from  sample  to  sample,  according 
to  the  writer's  skill  level. 

Another  important  source  of  error  along  these  lines  is  the 
manner  in  which  terminology  originates.  The  creation  of  new 
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terms  and  the  use  of  old  terms  in  new  ways  are  a  normal  part  of 
everyday  speech  communication,  and  are  hcibitual  unconscious 
behavior.  Relatively  few  persons  seem  to  have  cultivated  an 
awareness  of  their  normal  tendency  to  behave  in  such  a  manner, 
but  doing  so  is  essential  to  becoming  a  good  technical  writer. 
Each  term  used  in  technical  documents  must  be  traceedsle  to  a 
single  authoritative  source  so  that  every  reader  will  associate 
the  same  meaning  with  the  term.  Otherwise,  the  term  may  be 
misunderstood.  Even  a  writer  who  is  aware  of  the  terminology 
problem  may  be  confounded  by  the  large  nvimber  of  terms  present 
in  specifications  and  may  inadvertently  use  one  in  a  confusing 
manner.  Some  progress  towards  a  means  of  making  specification 
writers  aware  of  such  potential  misunderstandings  is  reported 
under  "Remapping  of  information, "  in  Section  III  below. 

Other  ways  in  which  language  is  often  misused  in  specifications 
are  covered  in  Section  III  under  "Usefulness  of  syntactic 
information. " 


e.  Inconsistencies 

Inconsistencies  are  among  the  most  difficult  errors  to 
identify,  since  doing  so  in  an  exhaustive  manner  would  require 
checking  each  specification  requirement  against  all  the  rest 
for  expressed  as  well  as  implied  conflicts.  They  usually  occur 
because  of  the  large  size  of  the  document,  and  human  inability 
to  remember  the  entire  content.  Another  cause  of  inconsistency 
is  the  use  of  inconsistent  terminology,  called  elegant 
variation,  which  is  taught  in  the  schools  as  desirable  in  other 
forms  of  writing,  but  which  is  unwise  in  technical  and  legal 
writing.  Some  progress  on  a  method  that  makes  a  large  class  of 
inconsistencies  easier  to  find  is  reported  in  Section  III, 
under  "Remapping  of  information." 


f.  Poor  Business  Practices 

The  use  of  what  are  considered  poor  business  practices  is  one 
of  the  simplest  of  the  seven  categories  to  identify  in 
practice.  There  are  four  types  that  are  often  erroneously  used 
in  contracts.  These  are: 

(1)  Specifying  agreements -to -agree  (NAVMAT,  1978) , 

(2)  Warranting  suitability  of  an  item  for  a  specific 
purpose  (ASBCA,  1967) , 

(3)  Specifying  by  vendor  and  part  number  (ASBCA,  1967) ,  and 
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(4)  Specifying  design  when  specifying  form,  fit  and 
function  would  suffice  (Clough,  1975) . 


g.  emissions 

Omissions  from  the  generic  sections  of  specifications  are 
surprisingly  easy  to  catch.  An  experienced  reviewer  easily 
notices  that  certain  routinely  required  items  are  missing.  On 
the  other  hand,  missing  requirements  that  are  peculiar  to  the 
item  being  specified  are  very  difficult  to  recognize. 


Usage  errors  by  type 

Number  of  Defects 

shall/will 

27  xxxxxxxxxxxxxxxxxxxxxxxxxxx 

any 

23  xxxxxxxxxxxxxxxxxxxxxxx 

capable 

22  xxxxxxxxxxxxxxxxxxxxxx 

or 

20  xxxxxxxxxxxxxxxxxxxx 

area 

9  XXXXXXXXX 

select 

8  XXXXXXXX 

insure/ensure 

8  XXXXXXXX 

comprise 

7  XXXXXXX 

etc . 

7  XXXXXXX 

various  other 

words  6  XXXXXX 

with/for 

3  XXX 

focus 

2  XX 

include 

2  XX 

all/each 

1  X 

Figure  1.  Relative  frequencies  of  word- related 

errors 

in  a 

75 -page  sample  of  specification 

text . 

9 


NAWCTSD  TR  93-022 


DCFECT  TYPE 


UMDBtLTlNG  CMISES 

Piecing  docueents  together  froei  Multiple  sources 
Lack  of  editing 

Changes  Made  to  doctMwnt,  nuMbers  not  updated  globally 
Lack  of  use  of  available  software  tools 
IndiscriMinate  copying  froM  boilerplate 
Failure  to  read  the  cited  doriMwnt 
Failure  to  check  revision  status  of  docuaents 
Lack  of  acronya  glossary  in  lar^  docuaent 
Writer  does  not  know  the  Meaning  of  the  acronya 
Inadequate  front-end  analysis 

Specifier's  technical  knowledge  is  inadequate 
Weak  verbal  skills 

Writer  does  not  understand  the  intended  req't 
Sentence  too  coMplex 
Non-coaaittal  attitude 

Overdependence  on  autoaated  spelling  checker 
Negative  training  in  verbal  skills 
ROTtE  in  production  contract 

InadeqiMte  training  in  contract  adsin. 
Paradigas  enciMber  hiMan  thought  process 
No  standard  outline  and  boilerplate 
I  Short  schedule 

I  I 

1  2  3  4  5  .  6  7  8  9  10  .  11  12  13  U  15  .  16  17  18  19  20  .  21  22 


Wrong  section 
Paragraph  number ing 


Cross-references 
External  references 
Figure  references 
Undefined  acronyms 


Subject  matter  wrong 
Logically  impossible 
Impossible,  other 
NvMierical  requirements 
Illegal 


Incorrectly  stated 
Non-specific  words 
Nonsense 

Grammatical  errors 
Incorrectly  used  words 
Ambiguities 
Spelling  errors 


Self -contradictory 
Synonyms 


Agreement s- to- agree 
Warrant i ng  suitability 
Specifyii^  make  &  model 
Design  vice  performance 


X  X 
X 
X 

X  X 


Missing 
TBO's  in  text 


Table  I.  Types  of  specification  defects  related  to  their 
underlying  causes. 
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The  Uaderlylng  Reasons  for  Defects  in  Specifications 

The  22  reasons  described  below  are  suggested  for  the  occurrence 
of  the  above-mentioned  types  of  errors.  A  brief  description  is 
given  for  each.  These  are  correlated  with  the  error  types  in 
Tcd>le  I.  While  some  of  them  can  be  verified  by  siitple  deduction, 
many  of  the  reasons  listed  are  based  on  informal  observations 
made  in  the  course  of  professional  practice,  and  therefore 
deserve  only  the  status  of  untested,  but  carefully  chosen,  and 
probably  trxie ,  hypotheses . 

It  is  evident  from  Table  I.  that  the  dominant  causes  of  error  are 
lack  of  editing  and  weak  verbal  skills.  Upon  this  basis  the 
search  for  automated  editorial  tools  is  justified. 


a.  Piecing  together  documents 

Usually,  specifications  are  not  written  from  beginning  to  end 
by  a  single  person.  Instead,  the  lead  engineer  delegates  the 
writing  of  niimerous  sections  to  specialists,  who  may  not  be 
aware  of  the  overall  goals  of  the  project,  and  may  have 
parochial  views  about  certain  requirements.  The  lead  engineer 
is  faced  with  the  difficult  task  of  fitting  all  these  pieces 
together,  finding  all  the  places  where  they  may  conflict,  and 
adjusting  them  to  be  correct  and  consistent  with  each  other. 


b.  Lack  of  editing 

Most  published  matter  must  undergo  several  rigorous  rounds  of 
editing  before  it  gets  into  print.  Often  it  is  substantially 
changed  by  rewrite  editors.  Such  care  is  not  often  taken  with 
engineering  specifications.  In  most  engineering  organizations, 
comments  on  the  draft  are  sought  from  senior  engineers  and 
engineering  managers.  The  specifier  then  rewrites  portions  of 
the  document  in  response  to  the  comments.  Usually,  very  little 
is  altered  from  what  was  written  in  the  first  draft. 

We  believe  that  there  are  several  factors  at  work  diminishing 
the  value  added  by  such  an  engineer- only  editing  process. 

First,  editing  is  a  special  skill,  requiring  talent  and 
training.  While  a  few  engineers  may  have  the  talent,  even 
fewer  have  done  anything  to  cultivate  it.  Hence  it  is  unusual 
for  an  engineer  to  notice  and  flag  errors  in  grammar,  usage, 
and  consistency  that  a  skilled  editor  would  catch. 

One  possible  explanation  for  why  so  few  incorrectly  stated 
requirements  are  caught  is  that  all  of  the  reviewers  are 
persons  of  very  nearly  the  same  mind- set  as  the  specifier. 
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Their  paradigm  for  what  is  correct  in  specifications  is  the 
same  as  that  of  the  writer,  and,  by  virtue  of  nature's  design 
for  the  human  mind,  they  are  uncLble  to  recognize  many  defects 
that  might  be  easily  seen  by  a  person  of  different  mind- set. 

One  way  of  solving  this  problem  would  be  to  use  technically 
trained  reviewers  who  are  not  feuniliar  with  the  subject  matter 
of  the  specification.  Such  persons  must  be  given  sufficient 
time  to  acquire  the  knowledge  needed  to  fully  understand  the 
text  as  a  part  of  the  editing  job.  Doing  so  would  amount  to 
making  a  trial  run  on  the  specification.  Such  an  approach  may 
have  a  natural  limitation  in  that  the  reviewers  may  no  longer 
be  effective  editors  once  they  acquire  a  thorough  knowledge  of 
the  subject  matter. 


c .  Incoiq>lete  changes 

Often,  when  a  document  is  in  nearly  finished  form,  a  change  is 
made  in  a  single  location,  and  the  document  is  not  searched  for 
other  text  that  may  be  affected  by  the  change.  A  simple 
example  is  the  case  where  an  entire  paragraph  is  added  or 
deleted,  and  the  resultant  renumbering  sets  askew  all  of  the 
cross-references  that  refer  to  the  renumbered  paragraphs. 


d.  Lack  of  use  of  available  software 

Many  writers  of  specifications  are  not  aware  that  specialized 
software  is  available  off-the-shelf  or  can  be  easily  developed 
to  ease  their  workload  and  enhance  the  quality  of  their 
product.  Many  of  the  commercially  available  style  checking 
programs,  all  of  which  are  descendents  of  CRES  (Kincaid  et  al, 
1982)  allow  users  to  supplement  their  inspection  and  annotation 
rules.  PARANA  checks  paragraph  numbering,  cross-references, 
and  references  to  military  specifications  and  standards. 

ASSIST  (NAEC,  1986)  checks  the  specification  tree  to  the  fifth 
tier  for  currency.  As  well  as  these  application- specific 
tools,  there  are  some  little -used  features  in  the  available 
word  processors  that  can  eliminate  certain  types  of  errors.  An 
example  is  the  automatically  updated  paragraph  cross-references 
in  some  of  the  newer  word  processing  programs. 


e.  Indiscriminate  copying  from  boilerplate 

While  the  availability  of  specification  boilerplate  greatly 
enhances  the  writer's  productivity  and  the  quality  of  the  work, 
it  may  also  introduce  errors  because  of  the  blind  trust  that 
may  be  placed  in  it.  There  is  a  tendency  to  copy  the 
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boilerplate  in  its  entirety  into  the  document  being  prepared, 
and  not  tailor  it  to  the  specific  application.  All  boilerplate 
must  be  checked  for  relevance  and  for  consistency  with  the  rest 
of  the  document.  McRobb  (1989)  gives  an  example  in  which  a 
boilerplate  outline  has  been  applied  verbatim,  producing  a 
monstrous  result. 


f.  Failure  to  read  the  cited  doctuaent 

In  order  to  properly  cite  a  specification  or  standard,  a  writer 
needs  to  be  familiar  with  its  content-  A  classic  error  of  this 
sort  was  often  found  in  citations  of  DD-G-451D,  a  general 
specification  for  sheet  glass  products,  which  was  used  for  many 
years  until  it  was  cancelled  in  1986.  Many  specifications  that 
cited  it  said  merely  that  the  glass  had  to  conform  to  DD-G- 
4 5 ID.  Persons  who  actually  read  DD-G-451D  found  that  it  was  a 
general  specification  defining  the  grades  of  glass,  from  the 
most  flawless  plate  glass  for  mirrors  down  to  greenhouse- 
quality  glass  which  needed  only  to  be  translucent.  So  in  many 
cases,  unbeknownst  to  the  specifier,  contractors  had  the  option 
of  using  the  very  cheapest  grade  of  glass  available,  greenhouse 
glass,  regardless  of  the  application. 


g.  Failure  to  check  revision  status  o£  documents 

Even  though  the  specifier  may  be  familiar  with  the  content  of 
the  specifications  and  standards  to  be  cited,  a  current  index 
must  be  checked  to  determine  the  status  of  the  document.  Doing 
so  is  a  relatively  simple  clerical  task  and  may  be  delegated  to 
non- technical  personnel.  Failure  to  check  this  seemingly  minor 
detail  often  results  in  expensive  claims  from  contractors  who 
are  unable  to  meet  a  stated  requirement,  but  expend  a  great 
deal  of  effort  trying  to  do  so.  A  good  example  is  the  case 
where  a  contractor  spent  a  great  deal  of  time  and  incurred 
delays  trying  to  find  a  large  quantity  of  an  obsolete  part  that 
was  unwittingly  specified. 


h.  Lack  of  acronym  glossary  in  large  dociment 

The  normal  practice  in  preparing  documents  is  to  define  each 
acronym  the  first  time  it  is  used  in  the  text.  This  is  done  to 
be  sure  the  reader  will  not  be  puzzled  about  what  the  acronyms 
mean.  When  the  document  is  only  a  page  or  two  long,  it's  easy 
to  scan  backwards  and  find  the  places  in  the  text  where  the 
acronyms  are  defined.  In  a  hundred -paged  document,  it  may  take 
hours  to  find  just  one  definition.  One  solution  to  this 
problem  is  to  put  a  glossary  of  acronyms  in  the  document  where 
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it  is  easy  to  find.  It  is  relatively  easy  to  check  the  text  to 
make  sure  each  acronym  appears  in  the  glossary.  Specifications 
of  more  than  about  twenty  pages  should  always  have  an  acronym 
glossary.  Such  a  glossary  has  been  added  to  this  report  so 
that  the  author  cannot  be  accused  of  ignoring  his  own  advice. 


i.  Writer  does  not  know  the  meaning  of  the  acronym 

Often  an  acronym  is  so  overused  that  people  have  seen  and  heard 
it  enough  times  to  form  a  concept  of  its  meaning  while  never 
having  seen  it  formally  defined.  They  begin  to  use  the  acronym 
in  their  speech  and  writing  to  convey  their  own  concept  of  its 
meaning,  which  is  not  necessarily  the  meaning  intended  by  the 
person  who  coined  the  acronym. 

How  many  times  in  meetings  where  acronyms  are  being  bandied 
about  have  you  heard  someone  interrupt  the  discussion  with  the 
question  "Pardon  me,  but  I’m  new  to  this  topic.  Could  someone 
tell  me  exactly  what  XXXX  means?"  Usually,  no  one  in  the 
meeting  is  able  to  give  a  satisfactory  definition  of  the 
acronym,  and  all  but  one  person  looks  like  a  fool. 

Whenever  an  acronym  is  found  to  be  undefined  in  a  draft 
document,  the  reviewer  is  advised  to  checi:  all  usages  of  the 
acronym  carefully  for  errors  in  meaning,  because  if  the  drafter 
knew  the  meaning  of  the  acronym,  in  all  likelihood,  it  would 
have  been  defined  in  the  text. 


j.  Inadequate  front-end  analysis 

The  message  here  is  simple.  It's  tough  enough  to  write  good 
specifications  when  you  thoroughly  understand  what  is  to  be 
specified.  It's  impossible  when  you  don't. 


k.  Specifier's  technical  knowledge  is  Inadequate 

This  situation  is  similar  to  the  case  of  inadequate  front-end 
analysis.  Usually,  a  technical  specialist  is  available  to 
assist  in  the  writing  of  specifications  in  technically 
difficult  domains.  When  such  help  is  available,  the  defects 
are  caused  by  failure  to  use  the  available  help. 
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1.  Neak  verbal  skills 

Strong  verbal  skills  are  absolutely  essential  for  engineering 
specification  writing.  The  English  language  is  rich  in 
pitfalls  of  impreciseness  and  ambiguity.  The  linguists  Katz 
and  Fodor  are  reported  to  have  said  that  it  is  not  possible  to 
make  an  English  statement  that  is  totally  unambiguous 
{Obermeier,  1989) .  To  avoid  the  pitfalls,  specification 
writers  need  to  possess  a  large  inventory  of  alternatives  for 
expressing  their  ideas  and  always  choose  the  one  that  is  least 
likely  to  be  misunderstood.  Unfortunately,  the  verbal  skills 
of  engineers  are  not  cultivated  adequately  by  the  education 
system,  and  most  come  out  of  school  with  a  relatively  small  bag 
of  writing  tricks  (Vaughan,  1991) .  Engineers  are  usually 
perceived  as  people  who  communicate  mainly  through  graphics, 
and  spend  most  of  their  time  working  with  equipment.  Often,  as 
students,  they  look  upon  English  class  as  a  senseless  waste  of 
time  forced  upon  them  unnecessarily.  This  may  be  true  for  an 
engineer  who  wishes  to  spend  an  entire  career  doing  detailed 
design  work,  but  such  people  are  very  few  in  number.  While 
project  engineers  who  prepare  specifications  rate  well  above 
most  college  graduates  in  writing  skills,  there  is  still  a  need 
to  raise  those  skills  to  even  higher  levels. 


m.  Writer  does  not  understand  the  intended  requirement 

This  is  closely  allied  with  poor  front-end  work  and  lack  of 
technical  expertise,  but  can  happen  even  though  both  are 
excellent.  It  tends  to  happen  when  the  front-end  work  is  done 
by  a  different  person  from  the  one  writing  the  specifications. 


n.  Sentence  too  coiiq)lex 

Writers  of  specifications  tend  to  lose  sight  of  how  long  their 
sentences  are.  They  add  clause  after  clause  until  they've 
arrived  at  a  sentence  that  expresses  their  thoughts  exactly. 
Long  before  the  sentence  is  finished,  it  has  grown  so  complex 
that  it's  beyond  its  creator's  ability  to  comprehend  (Flesch, 
1946) .  Ignoring  for  the  moment  what  this  does  to  the  reader,  a 
sentence  so  built  often  contains  syntax  and  usage  errors.  This 
is  because  it  takes  too  long  to  read.  By  the  time  you  get  to 
its  end,  you've  forgotten  how  it  began.  Parsing  so  long  a 
sentence  takes  a  studied  effort,  and  no  one  but  an  English 
professor  or  a  lawyer  ever  does  such  a  thing.  The  reader,  of 
course,  approaches  the  text  with  no  prior  knowledge  of  what  the 
writer  intended.  Losing  his  or  her  train  of  thought  in  reading 
a  long  sentence,  the  reader  starts  over  and  tries  again,  and 
keeps  doing  so  until  the  text  is  understood.  When  the  sentence 
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has  an  error  in  grairanar,  the  reader  probably  will  not  be  adsle 
to  figure  out  what  the  writer  meant. 


o.  Non-committal  attitude 

Reluctance  to  make  binding  commitments  is  fundamental  to  human 
nature.  It  stands  to  reason  that  in  order  to  feel  confident 
when  making  a  commitment,  you  need  to  take  the  time  to  study 
what  the  commitment  involves.  Once  you've  done  so,  you're  sure 
you'll  be  able  to  live  up  to  your  word.  Specification  writers 
are  likewise  inclined  to  avoid  making  commitments  because  they 
usually  don't  have  the  time  and  information  to  do  the  studying. 
In  addition,  there  are  too  many  reasons  beyond  their  control 
why  they  may  not  be  able  to  live  up  to  their  commitments.  When 
they  do  make  a  commitment,  it  is  usually  accompanied  by  many 
qualifications  and  warnings. 

The  business  of  writing  engineering  specifications  runs 
contrary,  then,  to  our  very  nature  with  regard  to  making 
commitments.  Specifications  are  necessarily  definite  and 
concise.  Writing  them  involves  making  a  large  number  of 
binding  decisions  about  the  items  being  purchased. 


p.  Overdependence  on  automated  spelling  checker 

When  an  automatic  spelling  checker  is  used  to  check  a  document, 
it  does  not  read  the  document  and  understand  its  content. 

Hence,  when  it  finds  a  phrase  like  "pilot  icing"  where  the 
writer  meant  "pitot  icing",  the  error  goes  unflagged  and 
uncorrected.  Such  occurrences  were  found  to  happen  eOjout  once 
in  a  hundred  typed  pages.  They  may  be  extremely  damaging.  In 
one  specification  paragraph,  the  word  "preclude"  was  found 
where  "provide"  was  intended,  evidently  a  typo.  The  resulting 
sentence  was  a  clearly  stated  requirement  that  the  equipment  be 
built  so  it  didn't  work  properly.  It  went  unnoticed  through 
all  the  normal  reviews. 


q.  Negative  training  In  verbal  skills  as  taught  In  the  schools 

It  seems  incongruous,  but,  in  our  commercial  society,  the 
communications  skills  are  being  taught  from  the  point  of  view 
of  people  who  aspire  to  write  literature.  Hence,  some  writing 
practices  are  taught  in  English  class  that  do  not  apply  to 
specifications.  Such  teaching  creates  a  distinctly  different 
situation  than  that  discussed  under  "Weak  verbal  skills," 
above.  The  possession  of  certain  perfectly  good  skills  for 
literary  writing  may  interfere  with  the  production  of  high- 
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quality  specifications.  The  most  outstanding  one  is  avoiding 
the  monotonous  repetition  of  words  by  using  synonyms. 

Engineers  who  have  had  their  specifications  criticized  by  a 
lawyer  will  attest  that  keeping  the  terminology  rigidly 
consistent  is  far  more  important  than  writing  lively  prose. 
Another  often- taught  practice  is  the  use  of  sentences  that  vary 
in  structure.  Simple  declarative  statements  are  the  best  for 
specifications . 


r.  RDT&E  In  production  contract 

In  recent  years,  there  has  been  pressure  to  use  fixed-price 
contracts,  with  the  expectation  that  fixing  the  price  at 
contract  award  will  reduce  cost  growth.  This  practice  has 
forced  many  contracts  that  involve  extensive  engineering 
development  and  testing  and  the  normal  accompanying  uncertainty 
to  be  let  as  fixed-price  type  contracts.  Once  the  work  is 
under  way,  it  progresses  more-or-less  as  it  would  under  a  cost- 
plus  contract,  except  that  the  cost  growth  occurs  in  a 
contentious  and  uncontrolled  manner.  Very  few  engineering 
managers  seem  to  realize  that  they  are  fully  able  to  avert  this 
type  of  problem  by  taking  a  firm  stand  to  the  effect  that  the 
contract  involves  RDT&E,  and  should  be  funded  and  managed 
accordingly. 


s.  Inadequate  training  in  contract  administration 

There  are  a  few  topics  in  contract  administration  that  bear  on 
the  writing  of  engineering  specifications  that,  for  some 
reason,  aren't  being  adequately  taught  to  writers  of  engineer¬ 
ing  specifications.  This  lack  of  training  results  in  the 
Government's  use  of  widely  recognized  poor  business  practices, 
and  consequent  waste  of  money. 


t.  Human  thought  process 

The  engineer's  paradigm  of  the  equipment  being  specified  and  of 
the  specifications  themselves  tends  to  be  a  limiting  factor  in 
his  or  her  ability  to  perceive  specification  defects.  This  is 
a  specific  case  of  a  well-known  phenomenon  of  the  human  mind 
that  is  often  thought  to  be  caused  by  the  presence  of  adaptive 
filters  in  the  human  information  processing  system  (Dodd  and 
White,  1980) . 

The  filtering  problem  affects  the  writing  process  as  well  as 
the  editing  process.  Engineers  tend  to  think  more  in  terms  of 
the  design  of  hardware  and  software  rather  than  in  terms  of  its 
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desired  performance.  It  is  very  hard  for  someone  who  has  been 
trained  to  think  in  terms  of  hardware  to  practice  restraint  in 
telling  the  designer  how  to  build  the  system,  and  confine  the 
specifications  to  a  description  of  what  the  equipment  has  to 
do.  This  limitation  may  arise  from  the  lesser  degree  of 
meaningfulness  present  in  the  terminology  of  performance 
specifications.  Further  discussion  on  this  topic  appears  below 
under  "IV.  ADDITIONAL  FINDINGS." 


u.  Failure  to  use  standardized  outline 

There  are  several  standardized  specification  outlines  in  use. 

It  is  important  to  choose  the  one  usually  used  for  similar 
work.  Good  judgement  must  be  exercised  about  how  to  tailor  it 
to  the  specific  application.  For  training  devices,  the 
standard  outline  is  taken  from  MIL-STD-490A.  Boilerplate 
specification  paragraphs  save  labor  and  promote  uniformity  from 
one  purchase  to  the  next.  Organizations  that  are  committed  to 
the  idea  of  providing  their  engineers  with  good  boilerplate 
specifications  have  also  developed  explanatory  material  to 
accompany  each  paragraph  of  boilerplate. 


V.  Short  schedule 

Lack  of  enough  time  is  the  most  often  -ised  excuse  for  failure 
to  do  a  good  job.  Surprisingly,  the  research  revealed  only  one 
type  of  defect  that  is  directly  attributable  to  short  schedule. 
That  was  cases  where  "To  Be  Determined"  or  "TBD"  was  found 
imbedded  in  the  text  of  a  supposedly  finished  document.  Other 
defect  types  may  be  indirectly  attributable  to  short  schedules, 
but  those  would  be  attributed  in  this  scheme  to  one  of  the 
other  reasons,  like  insufficient  editing. 
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III.  SOFTNjUtB  DEVELOPMENT 


Parsing  Sentences 

The  development  done  under  this  project  has  been  similar,  with 
one  important  difference,  to  the  ongoing  work  being  done  by 
commercial  software  houses  in  pursuit  of  a  credible  "grammar  and 
style  checker"  program.  Most  personal  con^uter  users  have  tried 
such  software,  and  many  have  decided  that  it  does  not  perform 
especially  well.  Often,  when  a  commercially  available  grennmar 
checker  generates  a  comment  about  syntax,  the  comment  is 
erroneous  (Bates,  1990) .  The  difference  between  such  software 
and  that  developed  under  this  project  lies  in  the  character  of 
the  text  being  analyzed  and  the  diversity  of  errors  pursued.  The 
grammar  and  style  checkers  search  for  a  great  number  of  possible 
faults  in  sentences  with  practically  no  bounds  to  their  meaning. 
On  the  other  hand,  the  specification- tool  software  seeks 
relatively  few  possible  faults  in  text  limited  to  the  relatively 
routine  language  of  specification  requirements. 

Another  difference  between  the  objectives  of  this  project  and  the 
mainstream  of  editorial  software  developers  is  best  understood  by 
contrasting  it  with  similar  exploratory  development  done  by 
Kieras.  Kieras  has  developed  advanced  editorial  software  that 
incorporates  a  parser  and  a  set  of  editorial  rules  with  the  aim 
of  helping  writers  produce  text  that  is  easy  to  comprehend 
(Kieras,  1990) .  Our  aim  has  been  to  help  writers  produce  text 
that  is  more  difficult  to  misinterpret,  which  is  not  the  same, 
since  readers  of  specifications  often  consciously  seek  alternate 
interpretations  in  an  effort  to  minimize  costs. 

After  determining  that  none  of  the  personal  computer  software 
packages  available  at  the  time  permitted  users  to  access  the 
needed  syntactic  data,  a  suitable  parser  was  developed.  The 
parser  itself  is  not  the  topic  of  primary  interest,  but  since  a 
good  deal  of  effort  went  into  its  development,  a  few  words  about 
it  are  in  order. 

The  parser  developed  as  part  of  this  investigation  is  best 
described  as  an  augmented  chart  parser.  The  name  "chart  parser" 
means  that  it  operates  first  by  building  a  table  of  all  the  well 
formed  substrings  (WFST)  in  the  sentence,  and  then  by  traversing, 
one  at  a  time,  each  possible  syntactic  tree  contained  in  the 
WFST.  "Augmented"  means  that,  while  building  the  WFST,  the 
grammatical  rules  are  not  applied  in  a  simple  rote  manner. 
Instead,  each  time  a  decision  is  made  to  apply  a  grammatical 
rule,  the  parser  is  given  the  option  of  executing  a  special 
subroutine  that  examines  numerous  aspects  of  the  situation  before 
deciding  whether  or  not  the  rule  should  apply.  The  grammatical 
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rules,  then,  are  applied  in  a  more  con^rehenslve  manner  than  they 
would  be  in  a  simple  context-free  parser. 

The  parsing  decisions  are  based  on  information  drawn  from  a 
lexicon,  which  contains  an  entry  for  each  recognized  word.  The 
lexicon  was  built  from  a  pre-existing  one  by  incorporating  new 
data  regarding  the  parts  of  speech  that  each  word  may  take,  as 
well  as  a  variety  of  other  information  of  use  in  the  decision 
process.  The  lexicon  of  about  5400  words  was  originally  compiled 
by  reducing  several  complete  sets  of  specifications  to  word  lists 
and  merging  the  results.  Information  on  the  parts  of  speech  was 
extracted  from  a  purchased  data  base,  and  then  reworked  manually, 
using  a  learner's  dictionary  as  a  source  for  additional 
information  regarding  parts  of  speech,  transitivity,  and 
countability. 

The  grammar  was  custom  built  to  parse  the  types  of  sentences 
found  in  specifications.  A  basic  set  of  rules  was  first  written 
using  some  published  grammars  as  examples  (Sager,  1981;  Mayer  and 
Kieras,  1987) .  Both  the  lexicon  and  the  grammar  were  fine-tuned 
for  specification  analysis  by  exercising  them  on  a  large  corpus 
of  specification  text,  and  revising  each  as  necessary  to  enhance 
the  accuracy  of  results  produced. 

Each  grammar  rule,  in  addition  to  the  rule  itself,  carries  a  flag 
that  is  used  to  create  a  pointer  to  the  central  word  of  each 
well -formed  substring.  For  example,  in  a  prepositional  phrase, 
the  pointer  points  at  the  object  of  the  preposition;  in  a  noun 
phrase,  it  points  at  the  main  noun.  This  feature  was  added  to 
permit  the  software  to  reduce  the  sentence  to  kernels,  which  were 
used  in  one  of  the  experiments  described  below. 

From  experience  in  testing  commercially  availedjle  grammar 
checkers,  it  appears  that  the  syntactic  parser  is  performing  at 
least  as  well  as  the  parsers  used  in  the  best  of  such  packages. 


The  usefulness  of  syntactic  information 

The  original  motivation  for  implementing  the  parser  was  to 
resolve  uncertainties  encountered  in  flagging  words  that  may  be 
used  as  more  than  one  part  of  speech.  In  those  cases,  data 
extracted  from  sentences  by  the  parser  provide  a  more  certain 
determination  of  the  part  of  speech  for  each  word. 

It  was  somewhat  of  a  disappointment,  though,  that  relatively  few 
cases  occur  where  the  data  can  be  actually  used  productively. 
They  serve  to  eliminate  only  a  small  number  of  unnecessary 
comments- -a  proportion  far  smaller  than  what  would  justify  the 
complexity  of  the  added  software. 
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In  the  course  of  reading  literature  on  Natural  Language 
Processing  (NLP) ,  which  was  a  prerequisite  to  programming  the 
parser,  discussions  were  found  of  syntactic  ambiguity  that 
contained  more  detail  than  those  in  any  known  source  on 
composition.  It  seems  surprising  that  such  knowledge  gained  in 
the  study  of  linguistics  has  not  been  carried  over  into  rhetoric, 
but,  by  all  accounts,  that  does  indeed  seem  to  be  the  case.  The 
appropriate  action  was  to  make  a  list  of  syntactic  forms  that 
might  be  of  concern  to  specification  writers  because  of  the 
ambiguities  they  may  generate.  A  set  of  example  sentences  was 
also  developed  to  illustrate  each  member  of  the  list  so  it  could 
be  understood  by  the  non-grcunmarian  engineers  in  the  intended 
audience.  That  list  is  documented  in  Appendix  B. 

Because  the  list  contained  types  of  ambiguous  syntax  that  were 
not  positively  identifiable  by  the  computer,  it  was  shortened  to 
the  forms  that  were  positively  identifiable,  and  tests  for  them 
were  implemented  in  the  software.  The  syntax  types  removed  from 
the  list  were:  "ellipsis  and  gapping,"  which  causes  the  parser  to 
report  that  it  has  encountered  a  sentence  fragment,  and  "adjacent 
phrases  lacking  a  clear  boundary, "  which  the  parser  erroneously 
bundles  together  as  a  single  phrase. 

Examining  the  specification  corpus  with  the  help  of  the  new 
software,  many  opportunities  for  ambiguity  that  formerly  escaped 
notice  became  evident.  The  most  commonly  encountered  among  the 
forms  that  generate  ambiguity  are  the  cases  where  a  modifier 
either  may  or  may  not  act  across  a  coordinating  conjunction. 
Examples  of  ambiguity  of  that  type  abound  in  nearly  every 
document  examined.  Another  type,  the  ambiguous  string  of 
prepositional  phrases,  seems  to  occur  in  specifications  mainly  in 
conjunction  with  the  phrase  "for  use..."  Therefore,  the  test  for 
it  can  be  implemented  very  easily  in  software  like  SpecTrE,  which 
has  no  parser. 


Semantics  of  surrounding  text 

As  mentioned  above,  an  investigation  was  done  to  gain  some 
knowledge  about  whether  or  not  semantic  information  contained  in 
surrounding  text  could  be  used  to  distinguish  relevant  flags  from 
irrelevant  ones.  Early  in  the  development  of  spec-writers' 
software,  the  testing  was  implemented  for  nearby  words  and 
phrases  to  reduce  the  number  of  erroneous  flags.  It  was  found 
that  approximately  ten  percent  of  the  erroneous  flags  could  be 
eliminated.  Tests  in  the  early  software  were  limited  to; 

a.  the  presence  of  certain  words  within  a  few  words  of,  or 
immediately  adjacent  to  the  flagged  item,  and 
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b.  presence  of  a  certain  word  somewhere  within  the  sentence 
being  examined. 

This  investigation  involved  using  the  parser  to  extract  the  noun 
phrases  from  the  sentence  and  the  paragraph  being  examined,  and 
making  them  available  for  use  in  deciding  whether  or  not  to  print 
a  conmient  when  a  sensitive  word  or  phrase  was  found. 

The  scheme  was  conceived  with  the  full  expectation  that  there 
would  be  sensitive  words  and  phrases  in  the  existing  ruleset  that 
were  not  sensitive  in  certain  cases  because  of  the  topics  being 
discussed  in  their  context.  However,  once  the  software  was 
implemented,  and  the  consequent  rigorous  definition  for  those 
cases  had  been  established,  the  search  for  actual  cases  in  which 
the  new  capability  could  be  used  was  found  to  be  fruitless.  The 
hypothesis  regarding  the  usefulness  of  semantic  information  in 
surrounding  text  is  untrue  in  terms  of  the  test  applied. 

As  a  result  of  the  ensuant  search  for  explanations  as  to  why  the 
hypothesis  was  untrue,  it  was  decided  that  the  rules  were  set  up 
mainly  to  flag  words  that  had  little  or  no  meaning  to  begin  with. 
Those  sensitive  words  that  had  definite  meaning  already  had 
exception  rules,  and  didn't  need  additional  ones.  In  cases  where 
a  "meaningless"  word  had  an  exception  nale,  the  exception  rule 
tested  for  a  definite  word  modifying  the  meaningless  one.  For 
example,  "high"  alone  is  meaningless;  "Eight  inches  high"  is  not. 
Furthermore,  the  sensitive  words  had  intentionally  been  chosen  as 
words  that  had  a  high  likelihood  of  being  troublesome  regardless 
of  where  they  were  encountered  in  a  set  of  specifications. 

This  line  of  reasoning  suggests  that  the  problem  ought  to  be 
approached  from  the  other  direction,  making  a  list  of  topics 
likely  to  appear  in  specifications,  and  then,  under  each  topic, 
make  a  list  of  words  that  might  be  sensitive  in  that  context. 

The  resulting  list  could  then  be  tested  on  the  corpus  to  see  if 
any  bugs  were  discovered.  Investigation  of  this  possibility 
remains  to  be  done.  One  possible  problem  is  forseeable  with 
rules  developed  in  such  a  manner:  they  will  probably  be  policy 
related,  and  likely  will  require  regular  maintenance. 


Re -mapping  of  information 

Two  other  possible  uses  of  the  parser  in  the  analysis  of  text 
were  investigated.  The  first  of  these  is  potentially  very  useful 
in  checking  l^rge  specification  documents  for  consistent  use  of 
terminology  and  consistency  of  requirements.  It  uses  the  parser 
to  identify  and  log  each  noun -adjective  group  found  as  it 
processes  the  entire  document  from  beginning  to  end.  Logged 
along  with  each  noun- adjective  group  are  the  paragraph  number. 
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the  sequence  number  of  the  sentence,  and  a  copy  of  the  entire 
sentence  in  which  the  noun -adjective  group  appeared. 

The  resulting  file  is  sorted  by  the  main  noun  of  each  noun¬ 
adjective  group,  with  subordinate  sort  keys  being  the  first  and 
second  words  of  the  group.  The  sorted  file  is  both  an  index  to 
the  text  and  a  re-mapped  copy  of  the  text  that  allows  the 
reviewers  to  examine  every  occurrence  of  each  term  in  close 
proximity  to  every  other  occurrence.  While  it  is  similar  to  the 
feuniliar  Key  Word  In  Context  (KWIC)  index,  there  are  two 
significant  differences.  First,  the  new  index  contains  entries 
based  only  on  noun -adjective  groups,  and  second,  each  entry 
contains  a  complete  copy  of  its  sentence  of  origin.  With  such  a 
listing,  a  sample  of  which  is  shown  in  Figure  2.,  it  is  not 
necessary  for  the  reviewer  to  remember  what  was  said  adsout  each 
topic  elsewhere  in  the  document. 

Since  the  mapping  is  one-to-many,  a  listing  of  this  file  may  be 
separated  into  sections  and  parcelled  out  to  several  editors 
without  sacrificing  the  completeness  of  the  examination. 
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Further  processing  was  done  on  the  list  mentioned  cQsove  in  search 
of  a  means  to  assist  editors  in  performing  a  terminology  audit. 
The  principle  is  single.  The  sorted  file  is  re- copied,  leaving 
out  every  line  whose  key  is  identical  to  the  one  before  it  or 
after  it.  The  result  is  a  list  similar  to  the  list  described 
above,  but  containing  terms  that  appear  only  once  in  the 
document . 

The  significance  of  being  used  only  once  is  that  such  terms  must 
belong  to  one  of  three  categories; 

a.  Fcuniliar  terms  that  are  readily  understood  by  a  broad 
audience, 

b.  Terms  drawn  from  a  cited  docximent,  and 

c.  Undefined  terms. 

The  editors,  of  course,  are  looking  for  the  undefined  terms, 
every  one  of  which  presents  an  opportunity  for  misunderstanding. 


Finding  nonsense  sentences 

The  final  experiment  performed  with  the  use  of  the  parser  was  to 
determine  its  usefulness  in  locating  "nonsense"  sentences. 
Nonsense  sentences  are  those  that  may  seem  authoritative  to  a 
casual  reader,  but,  upon  careful  study,  are  found  to  say 
something  different  from  what  was  intended,  and  may  make  no  sense 
whatsoever.  They  are  often  a  source  of  unintentional  humor. 

To  perform  the  experiment,  the  software  was  configured  to 
separate  from  each  sentence  the  main  noun  of  the  subject,  the 
main  verb,  and  the  object  of  the  verb.  The  software  effort  was 
kept  to  a  minimum.  A  great  deal  more  could  have  been  done  to 
enhance  the  number  of  correctly  parsed  sentences  without  a  major 
rework  of  the  parser.  The  objective  was  not  to  parse  correctly 
the  largest  number  of  sentences,  but  rather  to  generate  a  san^le 
of  kernel  sentences  that  could  be  examined  by  an  editor  in  search 
of  errors.  The  three  sentence  elements  were  printed  along  with  a 
copy  of  the  entire  sentence.  For  example: 

The  fuel  tank  area  of  the  airframe  skeleton  shall  be 
removed  and  the  airfreune  skeleton  shortened  by  the  fuel 
tank  area  length. 

reduced  to: 

area  shall  be  removed. 
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The  example  statement  is  nonsense  because  "area”  in  Standard 
Written  English  is  a  word  that  represents  an  abstract  concept  of 
measurement.  Area  cannot  be  removed.  The  writer  of  the  example 
sentence  erroneously  used  the  word  "area, "  in  an  informal  manner, 
evidently  for  want  of  a  recall  vocabulary  that  contained  more 
appropriate  words  like  "segments"  and  "sections." 

By  the  way,  misuse  of  the  word  "area"  by  engineers  is  a  recurring 
problem,  as  may  be  deduced  from  Figure  1.  One  case  that  was 
excimined  made  mixed  use  of  "area"  to  mean  both  "subject  domain” 
and  "geographical  region, "  intermingled  several  times  within  the 
same  short  paragraph.  Mixed  usage  of  that  sort  is  described  in 
books  on  rhetoric  as  a  tactic  used  by  unethical  writers  who  are 
intentionally  trying  to  lead  their  audience  to  an  erroneous 
conclusion.  Fowler  (1965)  called  this  tactic  "legerdemain  of  two 
senses."  One  should  not  be  surprised,  then,  that  a  contractor 
might  become  confused  when  usages  are  mixed  unintentionally. 

The  "nonsense  sentence"  experiment  involved  processing  a  sample 
of  285  sentences.  Of  that  seunple,  152  yielded  kernel  sentences 
in  which  the  subject,  main  verb,  and  object  were  identified 
accurately  by  the  parser.  Of  the  152  correctly  parsed  sentences, 
review  of  only  the  kernel  led  the  editor  to  36  errors.  The  133 
sentences  that  were  parsed  incorrectly  by  the  software  were  found 
by  a  human  parser  to  contain  eleven  kernel -related  errors. 

Although  it  affected  the  kernel -sentence  experiment,  failure  of 
the  parser  to  correctly  locate  the  main  kernel  of  the  sentence 
does  not  necessarily  make  the  syntactic  data  obtained  by  it 
useless  for  other  purposes.  Smaller  pieces  like  noun  phrases, 
prepositional  phrases,  and  infinitive  phrases  are  nearly  always 
parsed  correctly,  even  though  the  overall  structure  of  the 
sentence  may  be  missed.  The  correctly  parsed  fragments  are  made 
available  to  further  processing  steps,  so  correct  comments  are 
often  generated  by  the  program  even  when  the  parser  has  failed  to 
recognize  the  correct  structure  at  higher  levels  in  the  tree. 
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IV.  ADDITIQMXL  FINDINGS 


An  information  model  of  acquisition 

A  significant  portion  of  the  time  on  this  project  has  been 
devoted  to  learning  more  about  what  constitutes  error  in 
specifications,  finding  excimples,  and  determining  how  to  go  about 
avoiding  error.  A  wide  variety  of  sources  has  been  researched, 
and  the  information  gathered  is  being  used  in  the  development  of 
doctrine  and  instructional  materials  for  training  courses  that 
will  be  taught  to  NAWCTSD  engineers. 

Part  of  that  doctrine  is  the  information-processing  model  of  the 
procurement  process  that  was  presented  in  an  unpublished  paper 
prepared  for  the  Best  IR/IED  Paper  competition  (Oriel,  1992) . 

That  model  presents  the  procurement  process  as  a  communications 
system  with  certain  characteristics  that  are  significantly 
different  from  most  mechanistic  models  of  human  communication, 
but  nonetheless  feimiliar  to  the  engineers  in  the  target  audience. 


Operational  definitions  in  specifications 

Further  regarding  the  development  of  doctrine  on  specification 
writing,  another  part,  still  under  development  as  this  report  is 
being  written,  is  a  theoretical  treatment  of  the  meaningfulness 
of  specification  requirements.  Based  on  the  writings  of  the 
physicist  P.W.  Bridgman,  the  doctrine  being  developed  aims  to 
teach  engineers  to  depend  as  little  as  possible  on  the  power  of 
words  to  describe  requirements.  Instead,  they  are  to  be 
encouraged  to  state  requirements,  when  possible,  using  only  terms 
that  are  definable  as  the  results  of  a  specific  series  of 
"operations."  To  an  engineer,  "operations"  are  the  steps 
performed  in  the  construction  of  something,  or  the  steps 
performed  in  the  course  of  testing.  Operations  are  physical,  not 
verbal.  Bridgman's  doctrine,  called  "operationism, "  has  had  a 
profound  influence  on  both  the  natural  and  social  sciences  since 
its  introduction  in  1927  (Hempel, 1966) . 

By  Bridgman's  criteria,  design  requirements,  if  sufficiently 
complete,  constitute  $in  operational  definition  of  the  product. 
Popularly,  the  concept  of  operational  definition  is  described  in 
terms  of  the  statement;  "The  operational  definition  of  a  cake  is 
its  recipe."  Contract  law,  as  applied  to  engineering,  would  call 
that  recipe  a  "design  specification, "  which  is  a  method  of 
specifying  that  is  highly  compatible  with  every  engineer's  way  of 
thinking.  A  design  specification  is  one  that  tells  how  to  build 
the  required  item. 
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However,  Government  engineers  who  oversee  the  development  of  new 
products  are  discouraged  from  stating  their  requirements  in 
design  terms  because  contract  law  places  a  heavy  burden  of 
responsibility  on  parties  who  do  so.  Whenever  a  product  thus 
specified  does  not  perform  properly,  it  is  the  party  who  drafted 
the  design  specifications  that  must  bear  the  responsibility  for 
the  failure.  This  is  true  even  when  the  end- item  is  specified 
only  partly  in  design  terms.  In  that  case,  if  the  contractor 
chooses  a  way  to  complete  the  missing  parts  of  the  design  in  such 
a  manner  that  the  design- specif ied  parts  are  inadequate,  the 
contractor  is  not  likely  to  be  held  responsible  for  any  failure 
that  may  occur. 

The  alternative,  stating  requirements  in  terms  of  performance, 
however,  is  equally  risky.  Performance  requirements  are  merely  a 
natural  language  description  of  what  the  equipment  is  supposed  to 
do.  They  do  not  provide  an  operational  definition  of  the 
specified  item.  According  to  Bridgman's  philosophy,  they  can 
never  be  rigorous  unless  accompanied  by  a  set  of  testing 
requirements  that  are  sufficiently  complete  and  detailed  to 
constitute  an  operational  definition  of  all  the  terms  used  in 
describing  the  performance  requirements. 

With  its  newly  required  table  of  Quality  Assurance  (QA) 
requirements,  the  recently  circulated  draft  of  MIL-STD-490B,  the 
proposed  guiding  document  on  specification  practices,  has  moved 
nearer  to  an  operationist  doctrine  than  its  predecessor,  MIL-STD- 
490A.  The  format  that  MIL-STD-490B  gives  for  the  table  of  QA 
requirements  has  a  column  that  contains  a  general  statement 
regarding  the  test  method  to  be  used  in  verifying  each 
paragraph's  requirements.  Had  it  been  intended  to  achieve 
operationist  rigor,  that  column  would  have  to  contain  a  citation 
of  a  detailed  test  specification.  Examples  of  such  test 
specifications  abound  in  the  vast  literature  published  by  the 
engineering  standardization  community,  and  the  availability  of  a 
set  of  similar  procedures  for  testing  specific  items  to  be 
purchased  would  be  a  prerequisite  to  implementing  a  rigorous 
operational  approach  to  specifying  them.  One  category  of  goods 
that  may  presently  be  specif iced  in  accordance  with  operational 
definitions  is  the  semiconductor  devices  covered  by  the  MIL-S- 
19500  series  of  specifications  (SPAWARS,  1990) . 

It  is  appropriate  to  point  out  that  operationist  doctrine  has 
been  very  effectively  implemented  for  production  contracts  by 
means  of  the  strict  rules  regarding  the  use  of  the  first  article 
as  a  baseline.  Language,  after  first  article  acceptance,  is  no 
longer  the  basis  of  the  contractual  agreement  regarding  the 
configuration  of  the  product. 
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V.  COHCLnSIQNS  AMD  RBCOMMEMDATIONS 


Conclusions 

Appendix  C  of  this  report  sets  a  standard  for  elimination  of  the 
types  of  errors  commonly  encountered  in  specifications.  In  doing 
so,  it  fills  a  gap  left  by  existing  standards.  Considering  the 
stringency  of  Appendix  C,  one  may  readily  conclude  that  writing 
high-quality  specifications  is  a  task  that  requires  an  uncommonly 
high  level  of  literacy,  a  great  deal  of  specialized  training,  and 
the  help  of  cognitive  tools  like  those  developed  on  this  project. 


Recommendations 

Appendix  C  has  been  prepared  as  an  additional  step  towards 
training  engineers  to  write  high-quality  specifications. 

Appendix  C  sets  a  standard  for  elimination  of  the  types  of  errors 
commonly  encountered  in  specifications.  In  doing  so,  it  fills 
the  gap  left  by  the  existing  applicable  standards,  MIL-STD-490A 
and  MIL-STD-961C  (DMSSO,  1985) .  It  has  intentionally  been  kept 
short  and  to- the-point ,  and  is  written  using  plain  language.  It 
also  is  intentionally  written  in  the  first  and  second  person  to 
hold  the  reader's  attention. 

In  light  of  what  has  been  learned  on  this  project,  it  is 
recommended  that  the  research  effort  regarding  specification 
writing  be  conducted  on  a  continuing  basis.  It  is  further 
recommended  that  additional  research  be  conducted  to  apply 
emerging  technology  to  the  specification  quality  problem. 

Finally,  it  is  recommended  that  the  parser  software  developed 
under  this  effort  be  ported  to  the  Windows  environment  so  that 
syntactic  ambiguity  warnings  can  be  made  available  to  users  of 
Windows -based  software,  particularly  the  Joint  Acquisition 
Management  System  (JAMS) . 


Coordination  and  utilization 

Numerous  organizations  were  contacted  in  the  course  of  this 
exploratory  development.  Most  of  them  expressed  an  interest  in 
the  work,  but  no  others  were  found  to  be  conducting  directly 
related  efforts.  The  Government  agencies  furnishing  information 
that  was  used  in  the  course  of  this  work  were  the  Army  Logistics 
Management  Center,  the  Naval  Sea  Systems  Command,  the  Defense 
Standardization  Program  Division,  and  the  Los  Alamos  National 
Laboratory.  Academic  institutions  doing  likewise  were  the 
University  of  Michigan,  Texas  Tech  University,  and  the  University 
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of  California  at  Los  Angeles.  Points  of  contact  for  all  these 
organizations  appear  in  the  distribution  list  for  this  report. 

Much  of  what  has  been  learned  about  specification  writing  in  the 
course  of  this  research  has  been  carried  over  into  the  hypertext 
articles  embedded  in  the  software  product  called  SpecTrE. 

SpecTrE  is  presently  in  use  by  specification  writers  at  many  Navy 
commands.  The  content  of  the  text  articles  in  SpecTrE  has  been 
coordinated  with  the  departments  within  NAWCTSD.  A  Windows -based 
version  of  SpecTrE  has  been  prepared  and  is  being  used  as  part  of 
the  JAMS  program- -a  joint  Army,  Navy,  Air  Force  system  under 
development  to  help  facilitate  the  preparation  of  acquisition 
documents.  New  versions  of  both  the  SpecTrE  and  PARANA  programs, 
along  with  their  documentation,  are  in  preparation,  and  we  expect 
to  make  them  available  via  the  Defense  Technical  Information 
Center  in  1994. 
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GLOSSARY  OP  ACROmMS 

AGO  Administrative  Contracting  Officer 

AFSC  Air  Force  Systems  Command 

ASBCA  Armed  Services  Board  of  Contract  Appeals 

ASSIST  Automated  Specifications  and  Standards  Information  System 

CDRL  Contract  Data  Requirements  List 

CO  Commanding  Officer,  Contracting  Officer 

COTR  Contracting  Officer's  Technical  Representative 

ORBS  .omputer  Readcibility  Editing  System 

ZBD  Independent  Exploratory  Development 

IR  Independent  Research 

JAMS  Joint  Acquisition  Management  System 

KNIC  Key  Word  In  Context 

NAEC  Naval  Air  Engineering  Center 

NAVMAT  Naval  Materiel  Command 

NAWCTSD  Naval  Air  Warfare  Center  Training  Systems  Division 

NLP  Natural  Language  Processing 

PARANA  Paragraph  Analyzer  --a  computer  program. 

PCO  Procuring  Contracting  Officer 

QA  Quality  Assurance 

SOW  Statement  Of  Work 

SPAWARS  Space  and  Warfare  Systems  Command 

SpecTrB  Specification  (writers')  Trainer  and  Editor  --a 
computer  program. 

TBD  To  Be  Determined 

WPST  Well -Formed  Substring  Table 
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Appendix  K 

CXTR60RIBS  OF  SPBCIPICXTIQN  DBPBCTS 

1.  Organization  errors 

1.1  Requirements  stated  in  the  wrong  place 

1.2  Paragraph  nxiinbering  errors 

2 .  Reference  errors 

2.1  Errors  in  cross-references 

2 . 2  Erroneous  references  to  other  documents 

2.3  Erroneous  references  to  figures 

2.4  Acronyms  that  are  used,  but  are  not  defined  in 
the  text 

3 .  Incorrect  Requirements 

3.1  Subject  matter  is  wrong 

3.2  Statements  clearly  impossible  for  logical  reasons 

3.3  Statements  impossible  for  practical  reasons 

3.4  Errors  that  relate  to  numerical  requirements 

3.5  Violations  of  law  or  regulations 

4.  Requirements  Incorrectly  Expressed 

4 . 1  The  reviewer  can  deduce  what  the  writer  meant  to 
say,  but  the  text  says  something  else. 

4.2  Meaningless  words  like  "better",  "easy",  and  "as 
applicable"  that  leave  a  requirement  subject  to 
the  opinion  of  the  reader 
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4.3  Nonsense  -  reviewer  could  make  no  sense 
whatsoever  of  this  requirement 

4.4  Grammatical  errors  -  synteuc,  hyphenation, 

punctuation,  verb  tense,  subject -verb  agreement, 
parallelism  and  all  other  greunmatical  errors 

4 . 5  Incorrectly  used  words 

4.6  Ambiguities 

4.6.1  Semantic 

4.6.2  Syntactic 

4.6.3  Pragmatic 

4.7  Spelling  errors  -  may  be  clearly  misspelled  or 

often  a  misspelling  that  resulted  in  an 
incorrectly  applied  word  -  like  "pilot  ice"  vice 
"pitot  ice" 


5.  Inconsistencies 

5.1  The  text  contradicts  itself. 

5.2  Synonyms  used  without  being  clearly  defined  as 
synonymous . 


6.  Poor  Business  Practices 


6.1 


6.2 


6.3 


6.4 


Agreements - to - agree 

Warranting  suitability  of  an  item  for  a  specific 
purpose 

Specifying  by  make  and  model  number 

Specifying  design  when  specifying  performance 
would  suffice 


7.  Omissions 


7.1 


7.2 


Anything  that  the  reviewer  felt  belonged  in  the 
text,  but  was  not  there 

TBD's  (To  Be  Determined)  in  text  not  filled- in 
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Appendix  B 

SENSITIVE  WORDS,  PHRASES,  AND  STRUCTURES 
FOR  SPECIFICATION  WRITERS 


NBANINGLBSS  NOKOS 

The  following  words  are  meaningless  in  specifications  unless 
accompanied  by  words  that  convey  additional  information.  For 
excunple,  it  is  unlikely  that  the  meaning  of  "a  long  bolt"  could 
ever  be  agreed  upon,  but  "a  seven- inch  long  bolt"  is  clear. 


about 

high 

reasonable 

acceptable 

higher 

recognized 

accordingly 

highest  grade 

relevant 

accurate 

highest  quality 

reputable 

additional 

immediately 

safe 

adequate 

improper 

satisfactory 

adjustable 

instant 

secure 

af  fordcible 

insufficient 

securely 

significant 

appliceUDle 

irrelevant 

appropriate 

knowl edgeabl e 

similar 

average 

less 

single 

better 

long 

smooth 

careful 

low 

stcd>le 

convenient 

lower 

substantial 

deep 

major 

sufficient 

dependable 

multiple 

suitable 

desirable 

neat 

temporarily 

easily 

necessary 

tenporary 

easy 

normal 

the  like 

economical 

not  necessarily 

timely 

efficient 

optimum 

typical 

essential 

other 

uneconomical 

et  al. 

periodically 

unsafe 

etc. 

pleasing 

variable 

excessive 

possible 

various 

general  accordance 

practicable 

wide 

good 

practical 

workmanl ike 

greater 

proper 

quick 

worse 

worst 
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TOTALITY  WORDS 

Totality  is  a  concept  that  is  very  easy  to  state  in  words,  but 
very  rarely  occurs  in  the  real  world.  Totality  statements  are 
ed}undant  in  our  speech,  but  we  rarely  intend  for  a  listener  to 
take  them  literally.  Contractors  will  take  totality  statements 
literally  when  it  is  profitable  to  do  so.  The  following  words 
are  a  few  that  are  characteristic  of  totality  statements: 

all  entire  none 

always  identical  simultaneous 

continually  never 


IMMEASURABLE  WORDS 

These  words  are  mostly  nouns  representing  things  that  are  not 
generally  quantified. 

achievement  demonstration  realism 

best  _  practice  qualification  standard  practice 

confidence 


PRONOUNS 

Pronouns  are  always  risky  in  specifications,  because  readers 
often  find  a  different  antecedent  for  them  than  the  one  that  the 
writer  intended.  Pronouns  of  the  first  person  are  probably  less 
hazardous  from  the  standpoint  of  their  potential  to  be 
misunderstood,  but  are  considered  improper  in  specifications. 

it  they  we 

me  this  you 

them  us  your 


REQUIREMENTS  THAT  APPLY  TO  A  THIRD  PARTY 

The  following  words  are  often  used  in  such  a  way  that  the 
specifications,  when  taken  literally,  state  a  requirement  that  is 
not  binding  on  the  contractor.  Instead,  they  require  something 
of  the  individuals  who  use  the  equipment. 

by  the  instructor  operator  shall  trainee  shall 

instructor  shall  skill 
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WORDS  THAT  CAUSE  LEGAL  DIFFICULTIES 

The  following  five  groups  relate  to  legal  intricacies  of 
specification  writing. 

The  first  involves  something  called  "agreements  to  agree, "  which 
are  not  considered  a  good  contracting  practice.  Every  contract 
is  supposed  to  be  a  meeting  of  minds.  Requirements  to  reach  an 
agreement  at  some  future  date  are  contrary  to  the  legal  concept 
of  a  contract,  and  have  no  place  in  a  fixed-price  contract.  The 
following  words  are  characteristic  of  "agreements  to  agree": 

agree  concur  to  be  agreed  upon 


Public  Law  96-511,  The  Paperwork  Reduction  Act  of  1980,  restricts 
what  may  be  purchased  in  the  way  of  data  items  to  that  which  is 
described  in  authorized  Data  Item  Descriptions  (DIDs) .  The  law 
forbids  the  Government  from  expanding  the  scope  of  data  items 
beyond  that  described  in  the  DID  cited.  Therefore,  writers  of 
specifications  and  SOWs  must  be  very  careful  about  what  they 
require  in  connection  with  data  items.  In  general,  it  is  best  to 
avoid  discussing  data  requirements  in  specifications  and  SOWS. 
Such  discussion  belongs  in  block  16  of  the  form  DD1423.  The 
following  words  and  acronyms  are  characteristic  of  text 
describing  data  items: 

CDRL  DI  UDI 


The  following  words  often  appear  in  text  that  erroneously  grants 
a  warranty  to  the  contractor  regarding  information  contained  in 
the  specifications.  When  such  a  warranty  is  granted,  the 
contractor  may  rely  solely  on  whatever  the  Government  has  said, 
and  may  not  be  responsible  for  conducting  further  investigations 
before  proceeding  with  their  design. 

assure  guarantee  known 

certify  know  knows 

warrant 

The  following  words  are  often  found  in  text  that  assigns  duties 
to  individuals.  Such  assignments  should  not  be  buried  in  the 
details  of  specifications.  They  belong,  instead,  in  the  body  of 
the  contract,  and  should  be  drafted  by  a  contracts  specialist, 
not  by  an  engineer. 

buyer  contracting  officer  COTR 

CO  technical  representative  PCO 

AGO 
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The  following  words  often  appear  in  text  that  obligates  the 
Government  incorrectly  or  unnecessarily: 

furnish  loan  waive 

Government  Navy 


VERBS  AND  AT7XILLIARZES  THAT  DO  NOT  STATE  REQUIREMKHTS 

The  only  axixilliary  verbs  authorized  for  use  in  specifications 
are  "shall,"  "will,"  "should,"  and  "may."  Of  these,  ’^shall"  is 
the  only  one  that  may  be  used  to  state  a  binding  requirement. 
Requirements  stated  without  "shall"  are  either  not  binding  or  are 
erroneously  stated.  Each  occurrence  of  the  following  words  and 
phrases  should  be  checked  in  specifications  to  determine  whether 
or  not  a  requirement  has  been  misstated. 

are  is  to  must 

can  may  should 

is  might  will 


POTENTIAL  GENERATORS  OF  CONFLICTS 

The  following  words  and  phrases  are  likely  to  appear  in  text  that 
conflicts  with  other  portions  of  the  contract: 

contract  SOW  statement  of  work 

the  specification  unless  specified  herein 

price  as  specified  as  required 

commence  on  time 

unless  otherwise  specified 


HAZARDOUS  MATERIALS 

Hazardous  materials  are  sometimes  specified  inadvertently.  While 
it  is  not  possible  to  give  an  extensive  list  of  them,  the 
following  three  are  listed  because  they  are  likely  to  be  found  in 
training  device  specifications.  Also  significant  in  this 
category,  but  too  numerous  to  list  here  are  the  words  associated 
with  Class  I  Ozone  Depleting  Substances,  as  defined  in  the  Clean 
Air  Act . 

asbestos  beryllium  cadmiiom 
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AMBIGUOUS  WORDS 

The  following  three  words  are  often  used  in  a  military  context 
with  an  intended  meaning  that  is  different  from  what  is  seen  in 
normal  usage: 

activity  An  organization,  or  the  state  of  being 

active? 

operational  Pertaining  to  military  operations,  or  merely 

something  that  works? 

capability  It  may  be  "capable,"  but  not  actually  do  it. 

The  next  three  words  are  technical  terms  used  by  lawyers  as  well 
as  ordinary  words. 

consideration  This  may  mean  "something  given  in  payment." 

construction  This  could  mean  the  interpretation  of 

language . 

several  This  may  mean  "separate, "  instead  of  "more 

than  two . " 

The  following  are  words  in  general  usage,  and  are  either 
ambiguous  themselves  or  play  a  key  role  in  certain  frequent 
syntactic  ambiguities: 

any  Could  mean  several  or  only  one. 

anywhere  To  a  contractor,  this  could  mean  one  place, 

of  his  choice. 

as  well  as  Could  mean  "and"  or  "equally  well." 

because  This  phrase  may  introduce  either  a 

restrictive  or  nonrest rictive  dependent 
clause.  The  difference  between  the  two  uses 
is  distinguished  by  the  presence  or  absence 
of  a  comma  before  "because."  If  there  is  a 
comma,  the  clause  is  nonrestrictive.  The 
presence  of  the  comma  changes  a  requirement 
to  a  mere  observation.  Sabin  (1985)  has 
compiled  an  exhaustive  list  of  similar 
introductory  words,  and  gives  examples  of 
their  uses  in  both  restrictive  and 
nonrestrictive  clauses. 
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critical 

for  use 

include 

limited 

inflammable 


Could  mean:  important,  crucial,  analytical, 
dangerous . 

Who  does  the  using  and  who  does  the 
preparing?  This  phrase  often  generates  an 
ambiguous  series  of  prepositional  phrases. 

Is  this  word  followed  by  an  exhaustive  list 
of  everything  to  be  included,  or  by  only  a 
partial  list? 

This  word  is  often  used  by  spec  writers  who 
want  to  ask  for  something,  but  are  unable  to 
state  what  it  is. 

Use  flammable. 


noninflammable 


Use  nonf Icimmable. 


or 


problem 


property 


strict 


Contractor  may  choose  one  or  the  other. 
Expect  to  get  only  one  option. 

Does  this  mean  a  failure,  a  difficulty,  or  a 
training  exercise? 

Does  this  mean  real  property,  personal 
property,  or  an  attribute  of  something? 

Which  "strict":  "to  the  letter,"  or  "exactly 
as  intended"? 


uninflammable 


Use  nonflammable. 


up  to 
which 


More  than  the  minimum  is  a  gift  from  the 
contractor. 

See  "because." 


LOGIC  TRAP  WORDS 

These  words  are  often  found  in  specifications,  and  their  logic  is 
not  immediately  obvious : 

accept  or  reject  This  allows  no  option  for  partial  rejection. 

approve  It  is  not  wise  to  say  that  we  hold  something 

in  esteem.  Try  "authorize."  It  makes  less 
of  a  commitment. 
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designed  to 
either 

extent  possible 
maximum,  minimtun 

as  a  minimum 


It  may  be  "designed"  such,  but  will  it  be 
built  and  installed  that  way? 

Be  sure  you  don't  really  mean  "both." 

Often  found  after  "greatest"  or  "maximum, " 
this  may  require  an  unbounded  effort. 

These  two  words  may  denote  an  unbounded  or 
impossible  task.  Is  the  requirement 
practically  attainable? 

You  will  probably  not  get  more.  Contractors 
nearly  always  do  the  minimiun  required. 


ENGLISH  USAGE  ERRORS 


These  are  ordinary  English  usage  errors  that  are  likely  to  be 
found  in  specifications. 


affect 


call  out 
compliance  to 


Always  a  verb,  usually  means  "to  influence" 
or  "to  pretend  to  have  or  feel." 

Use  "cite,"  "refer  to,"  or  "reference." 

Use  "compliance  with"  or  "conformance  to." 


comprise 


This  word  is  usually  used  incorrectly.  Use 
"consist"  instead. 


effect 


ensure ,  assure 


insure 

irregardless 


Noun  for  "result"  or  "consequence, "  or  verb 
for  "bring  about"  or  "make  happen." 

"Ensure"  means  to  make  something  certain. 
"Assure"  means  to  remove  a  person's  doubt. 

"Insure"  should  refer  only  to  "indemnity." 

This  word  is  a  double  negative,  all  by 
itself . 


it's,  its 


"It's"  is  a  contraction  for  "it  is. 
is  the  possessive  form  of  "it." 


"Its 
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only  This  word  is  often  placed  in  the  sentence 

incorrectly.  Notice  how  the  following 
sentences  each  have  a  different  meaning: 

Only  the  dog  eats  dry  kibble. 

The  only  dog  eats  dry  kibble. 

The  dog  only  eats  dry  kibble. 

The  dog  eats  only  dry  kibble. 

The  dog  eats  dry  kibble  only. 

plan  Specifications  should  not  use  the  word 

"plan"  to  mean  "drawing." 

stationery  This  means  "writing  paper  and  supplies." 

"Stationary"  means  "immobile." 


OTHER  TROUBLESOME  WORDS 

The  following  words  fall  into  categories  too  small  to  warrant 
their  own  heading.  Each  of  them  is  nonetheless  a  potential 
source  of  error  in  specifications. 

align,  approximate  To  state  a  requirement,  these  words  must  be 

accompanied  by  an  explicit  statement  of 
tolerances . 


consider 


design,  develop 


DoD,  MIL 


establish 


host 


Requirements  containing  this  verb  often  give 
no  direction  regarding  action  to  take. 

Before  requiring  design  or  development, 
specification  writers  should  determine  that 
no  satisfactory  product  is  already  available 
off-the-shelf. 

Industry  standards  are  preferable  to 
Government  standards. 

This  word  should  always  be  accompanied  by 
clear  language  indicating  who  is  required  to 
do  the  establishing. 

When  referring  to  persons,  corporations,  or 
institutions,  the  one  referred  to  by  this 
word  is  reasonably  expected  to  pay  the  bill 
for  food  and  lodging. 
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note 

reference 

respond 

select 

standard 

TBD 

trade  off 


Notes  that  writers  have  left  to  themselves 
should  be  removed  from  docximents  before 
release. 

This  word  sometimes  appears  when  a  specific 
document  should  have  been  cited. 

When  referring  to  equipment,  this  word  is 
sometimes  indicative  cf  the  writer's 
tendency  to  personify  an  inanimate  object. 
Doing  so  is  not  conducive  to  clear  thinking, 
which  is  essential  to  specification  writing. 

The  context  should  clearly  indicate  how  the 
selection  is  to  be  made. 

This  word  should  refer  to  a  specific 
standardization  document.  It  cannot  stand 
alone  and  be  meaningful  in  the  context  of 
specifications . 

Never  release  a  document  that  contains  this 
acronym. 

study  This  phrase  should  be  accompanied  by  a 

description  of  the  study  to  be  done,  and 
what  action  is  to  be  based  on  the  results. 
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FAULTY  SENTENCE  STRUCTURES 

Some  defects  are  a  result  of  improper  phrase  or  sentence 
structure,  and  may  be  entirely  unrelated  to  the  meanings  of  the 
words  involved.  Those  types  of  syntax  are  illustrated  in  the 
examples  given  below.  Note  that  the  examples  given  are  in  no  way 
ungrammatical,  but  they  are  defective  in  the  context  of 
specifications  because  they  convey  alternative  meanings  depending 
upon  which  structure  the  reader  chooses  to  recognize. 

■  The  phrase  with  multiple  coordinating  conjunctions: 

.  .  .  gluing  and  clamping  or  riveting. 

Note  that  there  is  no  fixed  order  of  precedence  of  logical 
operators  in  English  as  there  is  in  formal  languages  like  Algol. 
This  example  is  a  very  simple  one.  The  really  serious  errors 
along  these  lines  are  to  be  found  in  long  sentences  that  have 
several  conjunctions. 

■  The  string  of  prepositional  phrases: 

.  .  .  predictions  shall  be  prepared  for  use  in  the  preliminary- 
design- review  meeting. 

Here,  the  reader  may  prepare  the  predictions  in  the  meeting,  much 
to  the  consternation  of  the  writer,  who  expected  to  use  the 
predictions  in  the  meeting.  Notice  that  the  meanings  of  the 
individual  words  are  exactly  the  same  regardless  of  which 
interpretation  you  prefer.  The  confusion  arises  because  the 
reader  has  no  clue  as  to  which  word  is  modified  by  each  of  the 
two  prepositional  phrases.  Strings  of  prepositional  phrases 
abound  in  specifications.  Most  are  disambiguated  by  semantic 
constraints.  Those  that  remain  ambiguous  are  very  difficult  to 
recognize  without  the  help  of  hints  from  the  computer  program. 

■  '  The  modifier  that  may  or  may  not  act  across  a  conjunction: 

. . .  stainless  steel  nuts  and  bolts. 

Specifications  using  this  phrase  require  only  that  the  nuts  be  of 
stainless  steel.  Bolts  of  any  material  would  be  acceptable, 
regardless  of  whether  that  is  what  the  specification  writer 
intended.  Some  linguists  classify  this  as  a  case  of  ellipsis 
since  the  words  "stainless  steel"  are  intended  to  modify  "bolts," 
but  their  presence  is  assumed  to  be  understood  by  the  reader . 
Errors  of  this  type  are  the  most  numerous  syntactic  ambiguities 
observed  in  specification  drafts,  and  they  deserve  special 
attention.  Formal  languages  solve  this  ambiguity  problem  by 
having  operators  that  force  the  logical  grouping  of  conjoined 
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symbols.  An  example  would  be  the  begin  ...  end  grouping  in 
Algol . 

■  Adjacent  phrases  that  lack  a  clear  boundary: 

The  completed  assembly  shall  be  designed  and  constructed  to 
withstand,  without  damage,  permanent  deformation,  cracking  or 
metal  fatigue,  stresses  incident  to  movement,  handling  in 
transit,  hoisting  amd  tiedown  aboard  transporting  vehicles,  final 
installation,  and  use. 

The  writer  in  this  case  probably  intended  a  semicolon  after 
"fatigue"  but  did  not  furnish  it.  That  punctuation  would  have 
provided  the  necessary  delimiter.  Better  form  in  so  complex  a 
situation  would  be  to  furnish  the  second  noun  phrase  as  a 
numbered,  indented  list.  Punctuation  often  plays  an  essential 
role  in  disambiguating  the  syntax  of  sentences  in  specifications. 
Commas,  hyphens,  and  semicolons  must  be  used  exactly  according  to 
the  rules.  Virgules  (slash  marks)  are  taboo. 

■  Ellipsis  and  gapping: 

The  generator  shall  supply  the  processor  with  10.5  amperes  and 
the  batteries  8.5  amperes. 

A  "shall  supply"  has  been  left  out,  and  there  is  no  indication 
whether  it  belongs  after  "and"  or  after  "batteries." 

■  Ambiguous  pronoun  referents: 

Prior  to  accepting  products  from  subcontractors ,  the  prime 
contractor  shall  evaluate  them  for  compliance  with  the  standards. 

Which  is  to  be  evaluated,  the  products  or  the  subcontractors? 
Flagging  these  in  specifications  is  very  easy.  We  simply  flag 
every  pronoun  and  direct  the  reader  to  make  sure  there  is  only 
one  noun  that  may  be  attached  to  the  pronoun. 
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Appendix  C 


RBCOMKEHDED  STANDASS 
FOR  THE 

PREPARATION  OF 
ENGINEERING  SPECIFICATIONS 


1.  SCOPE 

This  docximent  tells  engineers  and  scientists  some  things  they 
must  know  to  write  and  edit  high-quality  procurement 
specifications.  Most  of  the  material  covered  does  not  appear 
explicitly  in  any  other  readily  available  document.  We've  tried 
to  make  it  easy  reading  so  you  don't  get  tired  of  it  after  a  few 
pages.  Its  message  is  too  important  to  go  unread.  To  make  it 
easy  reading,  we've  made  the  language  less  formal  than  a  regular 
standard.  Its  outline,  however,  is  done  in  the  six -part  format 
of  a  procurement  specification  as  described  in  MIL-STD-490A.  It 
defines  certain  often-misunderstood  duties  of  both  specification 
writers  and  specification  reviewers. 


2.  APPLICABLE  DOCUMENTS 

The  items  listed  here  are  not  necessarily  cited  in  the  text  of 
this  specification.  They  are  all  relevant  to  your  work  as  a 
specification  writer.  Your  boss  should  have  a  copy  of  each  handy 
for  you  to  read. 

2 . 1  Government  Documrnts 

2.1.1  Specifications,  standards,  and  handbooks 

MIL- STD- 49 OA  -  Specification  Practices  (The  "B"  version  of 

this  standard  is  in  preparation) 

MIL-STD-961C  -  Military  Specifications  and  Associated 

Documents,  Preparation  of 

MIL-HDBK-245C  -  Preparation  of  Statement  of  Work  (SOW) 

2.1.2  Other  Government  documents 

Armed  Services  Board  of  Contract  Appeals.  Schaevitz 
Engineering . Inc .  ASBCA  No.  11524,  67-2  BCA  16678. 


C-1 


NAWCTSD  TR  93-022 


Naval  Material  Command.  Defense  Contract  Management  for 
Technical  Personnel.  Washington,  D.  C.  ,  1978. 

Oriel,  J.  T.  User's  Guide  for  SpecTrE.  Orlando,  Naval  Training 
Systems  Center,  1990. 

Oriel,  J.  T.  User's  Guide  for  CkList  and  PARANA.  Orlando, 

Naval  Training  Systems  Center,  1989. 

2 . 2  Non -Government  Publications 

Bly,  R.  W.  and  Blake,  G.  Technical  Writing.  New  York,  McGraw 
Hill  Book  Company,  1982. 

McRobb,  M.  Specification  Writing  and  Management.  New  York: 
Marcel  Dekker,  Inc.  ,  1989. 

Sabin,  W.  A.  The  Gregg  Reference  Manual .  New  York : McGraw- Hil 1 , 
Inc.  1985. 


3.  REQUIREMENTS 

3.1  Organization.  Organize  your  specifications  according  to  one 
of  the  outlines  in  MIL-STD-490  and  MIL-STD-961.  Try  to  follow 
the  outline  closely,  but  don't  let  the  outline  become  more 
important  than  clarity  and  conciseness.  McRobb  gives  an  example 
of  how  a  16 -page  specification  was  reduced  to  four  pages  by 
reworking  the  outline  to  better  fit  what  was  being  specified 
rather  than  by  following  the  prescribed  outline  (McRobb,  1989) . 

3.1.1  State  the  requirements  in  the  right  place.  Don't  put 
requirements  for  services  in  specifications.  Requirements  for 
services  belong  in  the  Statement  of  Work.  Specifications  are  for 
goods,  not  services.  Likewise,  don't  put  things  in  the 
specifications  that  belong  in  the  other  sections  of  a  contract. 
Specification  requirements  to  deliver  data  are  a  commonly  found 
violation  of  this  rule.  If  you  mention  a  data  item  in  the 
specifications,  and  expect  it  to  be  delivered,  you  must  also  have 
it  listed  in  the  Contract  Data  Requirements  List  (CDRL) .  You  must 
also  make  sure  that  each  specification  requirement  appears  under 
the  correct  heading. 

3.1.2  Paragraph  numbering.  Paragraph  numbers  shall  progress 
upwardly,  one  at  a  time,  so  that  there  are  no  gaps  in  numbering. 
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3.2 

3.2.1  Cross -  references .  When  the  text  of  a  specification  refers 
to  text  elsewhere  in  the  specification,  the  citation  shall  be 
correct.  Mistakes  in  cross-references  are  common  because 
paragraph  numbers  are  often  renumbered  and  the  cross-references 
neglected.  Such  mistakes  are  unacceptable  in  engineering 
specifications. 

3.2.2  References  to  other  documents.  When  you  cite  another 
document,  it  must  be  a  currently  active  document.  Canceled 
documents  shall  not  be  cited.  You  may  cite  obsolete  versions  of 
documents  only  when  you  are  specifying  modifications  to  equipment 
built  under  the  obsolete  version.  You  should  know  enough  about 
the  document  you're  citing  to  cite  it  correctly.  For  many 
specifications,  you  must  know  the  type,  grade,  alloy,  or  other 
parameters  that  must  be  specified  along  with  the  number  of  the 
doctiment.  Well -written  MIL-SPECS  will  have  this  information 
tabulated  in  the  section  entitled  "Ordering  Data"  or  "Acquisition 
Requirements" . 

3.2.3  References  to  figures  and  tadales.  Your  document  shall  not 
have  text  that  refers  to  the  wrong  or  non-existent  illustrations 
or  tables.  Likewise,  there  shall  be  no  illustrations  or  tables 
that  are  not  mentioned  in  the  text. 

3.2.4  Acronyms .  Acronyms  shall  be  defined  the  first  time  they 
appear  in  the  text.  For  documents  over  50  pages  long  that  use 
acronyms,  you  shall  also  include  a  glossary  of  acronyms. 

3.3  Requirements .  No  one  in  the  Executive  branch  of  the 
Government  has  the  authority  to  specify  goods  that  exceed  the 
Government's  minimxam  requirements.  The  Government  cannot  spend 
public  funds  to  suit  the  personal  preference  and  prejudice  of 
individuals.  You  must  specify  strictly  in  terms  of  what  the 
Government  really  needs. 

3.3.1  Correctness  of  subject  matter.  Statements  made  in 
specifications  shall  be  technically  correct.  If  you  must  specify 
something  you're  not  familiar  with,  seek  technical  advice  from 
someone  who  knows  the  topic  and  follow  the  advice. 

3.3.2  Logic .  Statements  that  you  make  in  specifications  shall  be 
logically  correct. 

3.3.3  Practicality.  Requirements  that  you  state  shall  be 
commercially  practical  to  implement'.  If  you  don't  know  a  way 
that  something  may  be  built,  find  an  engineer  that  can  advise  you 
on  the  matter  and  follow  the  engineer's  advice. 
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3.3.4  Numerical  requirements .  When  you  specify  things  by  nxomber 
you  need  to  take  particular  care.  See  to  it  that  your  numbers 
are  right.  Take  care  that  the  units  and  their  abbreviations  are 
correct  and  consistent.  Use  metric  units  unless  you  have  an 
important  reason  to  use  others. 

3.3.5  Precision  of  language.  Readers  of  your  specification  will 
not  always  know  ahead  of  time  what  you  meant  to  say.  When  the 
specification  says  something  similar  to,  but  not  exactly  what  you 
meant,  the  finished  product  will  be  something  different  from  what 
is  needed.  If  someone  notices  the  disparity,  fixing  it  will  cost 
both  money  and  embarrassment .  We  have  seen  many  cases  where  the 
language  used  in  specifications  is  so  poorly  written  that  even  an 
engineer  familiar  with  the  type  of  equipment  being  described 
could  make  no  sense  of  the  words  used.  While  such  occurrences 
are  OK  in  drafts,  they  are  unacceptable  in  released  documents. 

3.3.6  Specificity.  Requirements  shall  be  specific.  Don't  use 
words  like  "better",  "easy",  and  "as  applicable"  that  let  the 
reader  decide  what  they  mean. 

3.3.7  Grammar.  Specifications  shall  have  no  errors  in  grammar. 

We  usually  use  Sabin  (1985)  as  the  authority  on  grammar. 

3.3.8  Usage .  Word  usage  shall  be  as  defined  in  the  "Definitions" 
section  of  the  document.  Words  not  explicitly  defined  in  the 
specification  may  be  drawn  from  another  glossary,  provided  the 
glossary  is  cited.  Otherwise,  words  shall  be  used  as  defined  in 
your  Government  - issue  desk  dictionary.  If  the  dictiona^  has 
more  than  one  definition  for  a  given  word,  then  the  choice  of 
which  definition  to  use  will  be  made  by  the  contractor,  not  by 
the  Government . 

3.3.9  Clarity.  Statements  made  in  specifications  shall  have  only 
one  possible  meaning. 

3.3.10  Spelling.  You  shall  not  misspell  words  in  specifications. 

3.3.11  Consistency.  The  text  of  specifications  shall  not 
contradict  itself.  Tables  and  illustrations  shall  also  be 
consistent  with  the  text. 

3.3.12  Terminology.  Each  person,  place,  thing  and  idea  mentioned 
in  specifications  shall  be  referred  to  by  a  unique  name.  You 
shall  never  use  synonyms  without  clearly  defining  them  in  the 
"Definitions"  section  as  being  synonymous. 


C-4 


NAWCTSD  TR  93-022 


3.3.12.1  Definitions .  Wherever  possible,  terms  shall  be  defined 
without  dependence  on  interpretable  language.  Definitions  shall 
therefore  be  traceable  to  words  that  are  defined  in  terms  of  an 
empirical  procedure. 

3 . 4  Business  Practices 

3.4.1  Agreements- to-aaree.  For  fixed- price  contracts,  make  the 
requirements  of  specifications  complete  and  specific  so  that  the 
product  is  fully  defined.  There  shall  be  no  items  specified  in 
such  a  way  that  agreements  must  be  reached  with  the  contractor 
after  the  contract  is  awarded. 

3.4.2  Warranties .  A  warranty  is  an  assurance  that  something  is 

good.  When  we  warrant  an  item,  we  are  responsible  for  the  costs 
incurred  if  it  is  not  as  good  as  expected.  You  shall  not  write 

specifications  in  such  a  way  as  to  give  the  contractor  a 

warranty.  Engineers  often  unknowingly  grant  a  warranty  for  the 
correctness  of  technical  information  or  suitability  of  an  item 

for  a  specific  purpose.  A  statement  to  the  effect  that  something 

is  known,  appropriate,  suitable,  fit,  correct,  or  good  in  any 
other  way  is  sufficient  for  a  contractor  to  take  as  a  warranty. 
Given  such  a  warranty  by  the  Government's  engineer,  a  contractor 
may  rely  on  the  warranty  and  prepare  a  bid  with  no  further 
investigation  of  the  warranted  "fact" . 

3.4.3  Specifying  bv  make  and  model  number.  Specifying  by  make 
and  model  number  "or  equal"  should  be  done  only  when  you  are 
absolutely  certain  that  there  is  no  way  that  other  aspects  of  the 
design  can  be  done  in  such  a  way  that  the  specified  item  will  not 
work.  When  you  specify  by  make  and  model,  you  are  warranting  the 
suitability  of  the  item  for  that  purpose,  and  are  responsible  for 
the  outcome. 

3.4.4  Specifying  design.  Specifying  design  when  you  need  only 
specify  performance  is  foolish.  You  should  concentrate  your 
efforts  on  describing  the  needed  performance.  You  will  be  held 
responsible  if  the  design  details  you  specify  turn  out  to  be 
incompatible  with  the  design  done  by  the  contractor's  engineers. 

3 . 5  Completeness 

3.5.1  No  missing  requirements.  Specification  requirements  shall 
be  complete. 

3.5.2  TBDs .  Specifications  shall  not  contain  "TBDs"  (To  Be 
Determined)  unless  the  specification  is  for  equipment  to  be 
developed  under  a  cost-plus  type  contract. 
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4.  QUALITY  ASSURANCE  PROVISIONS 

This  section  tells  you  how  your  work  will  be  checked  to  make  sure 
it  meets  the  requirements.  All  errors  found  during  the  quality 
assurance  inspections  shall  be  corrected  before  the 
specifications  are  finalized. 

4.1  Inspection  of  organization.  Check  the  paragraph  titles  of 
the  draft  specification  against  the  standardized  outline.  If 
they  don't  follow  the  standard  outline  exactly,  the  author  must 
give  a  practical  reason  why  the  deviation  was  made. 

4.1.1  Requirements  stated  in  the  right  place.  Check  the  document 
for  requirements  that  properly  belong  in  the  SOW,  CDRL,  and 
Special  Provisions  by  carefully  reading  and  marking-up  a  hard 
copy.  Also  check  for  requirements  that  belong  in  a  different 
paragraph.  All  misplaced  requirements  shall  be  relocated  to 
where  they  belong. 

4.1.2  Paragraph  nxamberinq.  Use  the  program  PARANA  to  check  the 
paragraph  numbering  for  errors.  Correct  the  errors  found. 

4.2  Inspection  of  references 

4.2.1  Cross -  references .  Use  PARANA  to  help  you  do  an  exhaustive 
check  for  erroneous  cross  references.  Correct  the  errors  found. 

4.2.2  References  to  other  documents.  Use  PARANA  to  check  the 
references  to  other  Government  documents.  Look  up  each  one  to 
check  whether  it  is  current.  Correct  the  erroneous  references. 

4.2.3  References  to  figures  and  tables.  Use  your  word  processor 
to  search  the  document  for  all  occurrences  of  the  word  "figure" 
and  check  each  referenced  figure  for  relevance  to  the  text.  Also 
check  that  all  illustrations  are  mentioned  in  the  text.  Repeat 
the  process  for  tables.  Correct  all  erroneous  references. 

4.2.4  Acronyms .  Use  PARANA  to  tabulate  all  uses  of  each  acronym. 
Check  each  use  for  correctness.  Correct  all  the  errors  in 
acronym  usage. 

4.3  Inspection  of  requirements 

4.3.1  Correctness  of  subject  matter.  Technical  specialists  shall 

review  the  document  for  technical  correctness.  The  errors  found 
shall  be  corrected  be  the  document  is  finalized. 

4.3.2  Logic.  An  experienced  editor  shall  review  the  docxament  for 
logical  correctness .  The  errors  found  shall  be  corrected  before 
the  document  is  finalized. 
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4.3.3  Practicality .  A  systems  engineer  shall  determine  whether 
the  equipment  described  is  commercially  practical  to  build  and 
maintain.  When  it  is  found  that  the  equipment  described  is 
impractical,  the  specification  and  all  other  progreun  documents 
shall  be  rewritten  so  as  to  correct  the  problem. 

4.3.4  Numerical  requirements .  Technical  specialists  shall  check 
to  see  that  the  numbers  are  right.  An  editor  shall  check  that 
the  units  and  their  abbreviations  are  correct  and  consistent. 

The  author  shall  furnish  a  practical  explanation  if  other  than 
metric  units  are  used.  All  numerical  errors  shall  be  corrected. 

4.3.5  Precision  of  language.  Skilled  editors  shall  review  the 
specifications  with  the  help  of  computerized  editing  tools  like 
CkList  and  SpecTrE  to  find  imprecisely  stated  requirements. 
Imprecise  language  shall  be  corrected. 

4.3.6  Specificity.  Computerized  editing  tools  shall  be  used  to 
find  all  occurrences  of  non-specific  words.  Each  case  found 
shall  be  corrected. 

4.3.7  Grammar .  Skilled  and  experienced  editors  shall  find  all 
grammatical  errors  in  the  specifications.  All  grammatical  errors 
found  shall  be  corrected. 

4.3.8  Usage ♦  Skilled  editors  shall  review  the  specifications 
with  the  help  of  computerized  editing  tools  like  CkList  and 
SpecTrE  to  find  errors  in  word  usage.  Errors  in  word  usage  shall 
be  corrected. 

4.3.9  Clarity .  Statements  with  more  than  one  possible  meaning 
shall  be  found  by  skilled  editors.  Each  one  found  shall  be 
reworked  so  that  it  has  only  the  one  meaning  intended  by  its 
author . 

4.3.10  Spelling.  Misspelled  words  in  specifications  shall  be 
found  by  using  an  automated  spelling  checker.  Skilled  editors 
shall  review  the  specifications  to  find  improper  words  that  get 
by  the  spelling  checker  because  they  are  spelled  correctly  even 
though  they  are  not  used  correctly.  All  spelling  errors  shall  be 
corrected. 

4.3.11  Consistency.  Skilled  and  experienced  editors  shall  find 
all  contradictions  and  consistency  errors  in  the  specifications. 
All  consistency  errors  found  shall  be  corrected. 

4.3.12  Terminology.  Skilled  and  experienced  editors  shall  check 
to  see  that  there  are  no  inconsistencies  in  the  terminology  used 
in  the  specification.  The  editors  shall  be  equipped  with 
software  tools  to  enhance  their  effectiveness  at  checking 
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multiple  occurrences  of  each  term  used  and  at  checking  for  the 
erroneous  use  of  synonyms.  Each  instance  of  inconsistent 
terminology  shall  be  corrected. 

4 . 4  Inspection  of  Business  Practices 

4.4.1  Agreements -to- agree.  Skilled  editors  equipped  with 
computerized  tools  shall  inspect  the  specification  text  for 
statements  that  would  require  decisions  to  be  made  and  agreements 
reached  with  the  contractor  after  contract  award.  In  the  case  of 
specifications  to  be  used  with  fixed-price  contracts,  all  such 
occurrences  shall  be  removed.  In  the  case  of  cost-plus 
contracts,  the  agreements  shall  be  listed,  and  the  list  presented 
to  the  Contracting  Officer  for  concurrence  before  the 
specification  is  finalized. 

4.4.2  Warranties .  Skilled  editors  shall  examine  the 
specification  text  for  wording  that  would  grant  the  contractor  a 
warranty.  In  each  case,  the  wording  shall  either  be  corrected  so 
as  not  to  grant  the  warranty,  or  the  engineer  shall  present  proof 
that  the  warranty  will  not  result  in  an  expense  to  the 
Government . 

4.4.3  Specifying  bv  make  and  model  number.  Skilled  editors  shall 
examine  the  text  for  specification  by  make  and  model.  In  each 
case,  the  specification  shall  either  be  corrected  so  as  not  to 
grant  an  implied  warranty,  or  the  engineer  shall  present  proof 
that  the  warranty  will  not  result  in  an  expense  to  the 
Government . 

4.4.4  Specifying  design.  Senior  engineers  shall  examine  the  text 
for  design  specification.  When  cases  are  found  where  design  has 
been  specified  when  specifying  form,  fit,  and  function  would  have 
sufficed,  the  text  shall  be  corrected. 

4 . 5  Inspection  for  completeness 

4.5.1  No  missing  requirements.  Specification  requirements  shall 
be  checked  for  completeness  by  senior  engineers.  Items  found 
missing  shall  be  added  to  the  document. 

4.5.2  TBDs .  Specifications  shall  be  searched  for  the  strings 
"TBDs"  and  "To  Be  Determined".  Unless  the  specification  is  for 
equipment  to  be  developed  under  a  cost-plus  type  contract  all 
such  occurrences  found  shall  be  filled  in  with  properly  completed 
text . 
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5.  PREPARATION  FOR  DELIVERY 

This  section  is  not  applicable  in  this  case,  but  is  included  here 
to  preserve  the  customary  six-part  format. 


6.  NOTES 

6.1  Intended  use.  Documents  written  to  meet  this  specification 
are  for  use  in  the  procurement  of  goods.  Services  are  described 
in  Statements  of  Work.  Many,  but  not  all  of  the  provisions  of 
this  specification  are  applicable  to  Statements  of  Work  as  well. 
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